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PREFACE 

This book is based on a course of lectures on 
I have given in the Mathematical Laboratory 
bridge for several years. It is intended to be introductory, in the sense 
that no previous knowledge of the theory and practice of systematic 
numerical work is assumed, but it is not ‘elementary’ in the sense of 
using only school mathematics. It assumes familiarity with the calculus 
up to Taylor’s theorem and partial derivatives, acquaintance with 
differential equations and, in the chapter on linear simultaneous algebraic 
equations, with some of the simpler properties of matrices. But in all 
these cases what is wanted is mainly an understanding of the ideas 
involved rather than technical facility in manipulating algebraical or 

analytical expressions. . 

I have deliberately tried to restrict the algebraical and analytical work 

to the treatment of those methods which are useful in practice when 
numbers are substituted for the literal symbols of a general treatment, 
and to avoid developments which are of 7 purely formal interest. Such 
developments may be elegant mathematics, or may make the formal 
presentation more complete, but they are not contribut ions to numerical 
analysis, and are distracting rather than helpful to the reader who wants 
practical information about what calculations to make to obtain the 
results he requires, and how to carry them out. For a similar reason, 
I'have tried to give prominence to the importance of checking in numeri¬ 
cal work. Mistakes can occur in such work, and it follows that some 
process of checking is necessary to ensure that any results obtained are 
not vitiated by undetected mistakes. A treatment ol numerical met hods 
which does not pay some attention to this aspect of the subject seems to 
me to be quite unrealistic. In some worked examples I have deliberately 
introduced mistakes in order to show how, by suitable checking proce¬ 
dure, they can be detected, diagnosed, and corrected. 

For similar practical reasons I have deliberately omitted some 
examples of numerical work which seeni to have become almost classical, 
for example the evaluation of an approximation to \n by application ot 

Gregory’s quadrature formula to the integral J dx/( 1+* 2 ). This par¬ 
ticular calculation I regard as an example of how not to do numerical 
work; not because the method is wrong, but because it is not the most 
suitable for obtaining the required result (sec§ 6.51); use of it is therefore 
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an example of bad practice, and should not be presented as if it were an 
example of satisfactory numerical procedure. Much could be mitten on 
how not to do numerical calculations. I have mentioned in the text some 
procedures which should, in general, be avoided, mostly because thej r are 
not the most suitable for obtaining the results sought, but some because 
they are dangerous in that, if used without precautions, they may give 
wrong results. I have refrained from giving numerical illustrations of the 
dangers of such methods, except in one case (§ 5.81) in which it seemed 
advisable to give a warning, by means of a horrid example, of the dangers 
of a method for inverse interpolation which might at first sight appear 
attractive, and which has in fact been given in print as a usable method 
without any mention of the dangers. 

An introductory treatment such as that of this book cannot cover the 


subject completely; several of the chapters from Chapter IV onward 
could well be expanded to form a volume each. In particular, this book 
hardly begins to touch the needs of the specialist or research worker in 
the subject of numerical analysis; its purpose is rather to give an intro- 
duet ion to t lie subject to workers in other fields of pure or applied science 
who may have to carry out calculations of a non-trivial magnitude. In 
such contexts the accuracy of the approximations or measurements 
underlying the calculation to be done do not usually justify working to 
a greater aecuiaev t hail six or seven figures, and often a smaller number 
will he auequnte. On the other hand, the amount of calculation to be 
done to this accuracy may be considerable. In so far as the appropriate 
method lor carrying out a calculation depends on the number of figures 
kept in it. emphasis is therefore placed on methods suitable for calcula- 
tmns nl suhslanlial extent to moderate accuracy, rather than for a few 


calculations h, many figures. My personal experience of such work 
oxU ml.s o .-r 35 years, and most of this work has been concerned with 
calculations involving numerical approximations to some of the limiting 
processes of analysis. in particular integral ion (including the integration 
° f ' ‘lotions). On the basis of this experience, 1 believe this 

to be one o] the most important practical fields for the use of numerical 
methods, and have deliberately given it considerable prominence. 

Ju the past few years there has been considerable development of 
ogh-speed automatic general-purpose digital calculating machines. This 

, (V ,I ' nK : I,< has - 1 von P ,actit ”‘l importance to the study of the process 
0, 'f•K olMions for these machines, a process ustiallv referred 
‘"“j y :"'' ‘ I r,1:, ! n: - 1 - fctudy can be regarded as a branch of numerical 
‘ “Hu- . ml ;i i> one which is likely to grow in importance as more of 
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these machines become available. I have therefore included an intro¬ 
ductory chapter on this subject (Chapter XII). There are several systems 
of programming, and for brevity in an introductory account such as this, 
and to avoid confusing the reader with a number of alternat ives, it seems 
best to adopt one particular system in presenting illustrative examples. 

I have adopted the one with which I myself am most familiar, and which 
I think is one of the simplest to follow. But this must not be regarded as 
more than a means for illustrating in a simple form some of the general 
ideas involved in programming. For various reasons, some of which are 
mentioned in § 12.8, numerical methods which are convenient for hand 
calculation with the assistance of a desk machine are not the most 
suitable for an automatic machine, and vice versa. Brt this book is 
intended to provide an introduction to numerical analysis for those who 
will mainly be concerned with methods suitable for hand calculation, and 
little or no reference is made to other methods some of which might be 

more suitable for automatic machines. 

In the Bibliography I have included, as suggestions for further reading, 
some books and papers not referred to in the text, but I have not at¬ 
tempted to compile a complete bibliography of numerical analysis, or to 
give references to the history of the subject; the reader who is interested 
in early references should consult The Calculus of Observations by Whit¬ 
taker and Robinson. 

On matters concerning the use of desk calculating machines, I am 
conscious of a considerable debt to the late Dr. L. J. Comrie; the processes 
of §§2.25, 4.45, 4.46, and Example 7 I learnt from him, though whether 
he originated them I do not know; and there may well be other examples 
of his influence of which I am unconscious. Some of Dr. Comrie s long 
and varied experience in numerical work is incorporated in Chambers's 
Six-figure Mathematical Tables, but it is much to be regretted that he 
did not live to write a fuller work on the art of numerical calculation. 

In the derivation of central-difference interpolation formulae, I have 
followed a treatment which I learnt from J. G. L. Michel, and in the 
examination of truncation errors of interpolation and integration 
formulae I have followed a treatment which I learnt from Professor 
W. E. Milne while I was serving as Acting Chief of the Institute for 
Numerical Analysis of the U.S. National Bureau of Standaids. 

I wish to express my thanks to Dr. J. Howlett, of the Computing 
Section, A.E.R.E., who read the first draft typescript of this book, and 
to Mr. A. S. Douglas, who read the proof sheets, for many valuable 
comments and suggestions, and for a number ol corrections. Also l u ish 
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to thank Mr. P. Farmer for the photographs from which the drawings 
for Figs. 1, 2, and 3 were made, Mrs. Valerie Taylor for making the 
drawings themselves, my daughter for her skill in typing much of the 
text, and Dr. M. V. Wilkes, Director of the Mathematical Laboratory, 
Cambridge, for permission to avail myself of the services of Mr. Farmer, 
Mrs. Taylor, and my daughter, all members of the staff of the Laboratory. 
It is a pleasure also to thank those members of the staff of the Clarendon 
Press who have been concerned with the production of this book. 

D.R.H. 

CAVENDISH LABORATORY 
CAMBRIDGE 

May 1952 


ADDENDUM TO PREFACE 


With the co-operation of the Clarendon Press, I have taken the oppor¬ 
tunity, provided by a new' impression of this book, to make a number 
of corrections and two substantial alterations. Two sections (§§ 5.61 and 
3.53) have hem completely rewritten, as experience of calculations of 
the kind considered in these sections has led me to the conclusion that, 
for the readers for whom this book is mainly written, the methods now 
given are preferable to those given in the lirst impression. 

1 wish to thank various correspondents, and particularly Mr. R. E. 
Heard, Mr. 1>. R. Bland, and Mr. (1. A. Erskine, who have written to 
draw my attention to points requiring correction or modification. It is 
a pleasure to renew my thanks to the staff of the Clarendon Press for 
t heir co-operation. 


CAVENDISH L A B O R A TORY 
' A M B R 1 D U fl 
JULY 195.1 
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INTRODUCTION 


1.1. What numerical analysis is about 

The subject of numerical analysis is concerned with the science and art 
of numerical calculation, and particularly with processes for getting cer¬ 
tain kinds of numerical results from certain kinds of data. The following 
are some simple typical problems for which we may require processes for 

obtaining numerical solutions: 

(i) Tabulate (sinhz-z)/r> to five decimals for z = 0(0-l)3.t 

(ii) Given such a table, 

(а) find, as accurately as possible, the value of x for whic 

(sinhx—x)/x 3 = 0-2; 

(б) construct a table at intervals of 0-02. 

(iii) What values of x, y , z satisfy the equations 

xyz = S, x 2 -t/ 2 +z 2 = 6, x-f 2y+3z = 1°- 

(iv) Tabulate J dw for x = —2(01)2. 

(v) For what values of A has the equation 

y'+{ A-e-*> = 0 

got a solution for which y - 0 as z - ±co and for which 

J y 2 dx = l, 

— CO 

and what are the corresponding solutions ? 

Although from the point of view of numerical analysis the end to be 
attained is always a numerical result or set of results, the subject« not 
concerned with the results, that is to say answers to specific problem 
themselves but with the processes by which those results can be obtained. 
Md although the end is a numerical result, algebra and analysis are in- 

and the arguments by which they are derived, are general and indep 
dent of the particular values of the numbers to which they may 

t This is a standard notation for 'from x = 0 to 3 inclusive a. intervals of 0-1 m x'; 
Bee § 2-3. 
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applied, the subject may properly be regarded as a branch of mathe- 
matics.f But the algebra and analysis must be aimed at providing or 
establishing practical methods of obtaining numerical results ; otherwise 
it may be elegant mathematics, but is not a contribution to numerical 

analysis. 

This emphasis on practicable numerical processes requires a considerable 
change in attitude from that of ordinary algebra and analysis, to which 
the idea is quite foreign. Algebraical or analytical results which are 
formally complete answers may be almost or quite useless for numerical 
purposes. Consider, for example, the solution of a system of simultaneous 
linear algebraic equations. Any textbook of algebra shows how this can 
be expressed in terms of ratios of determinants, and this result is often 
presented in a form which seems to imply that there is nothing more to 
be said on the subject. But direct evaluation of the solution in this form 
is certainly not the practical answer to the problem of finding a numerical 
solution of a set of simultaneous equations. 

As another example, consider the solution of 

~ = l — Uxii, y — 0 at x = 0. (1.1) 

d.c 


I'he standard textbook treatment gives 

>/ — e~*' j e x ' dx, 


( 1 . 2 ) 


1> 


and regards t! i as a complete answer. And so, for numerical purposes, 
it is. proi i<l> I one has a table of i c jl dx. But in order to obtain such a 


0 


table is mueli easier to reverse the process and solve the differential 
•‘M’talion (1.1) by numerical Me thods, and then to evaluate the integral 
by udng (1.2; in the form 

I e r * dx --- c x ‘y t 


0 


than to evduati | « J ' dx numcricalK directly. Again the formal text¬ 
book answer is ot no practical use if numerical results are wanted in the 
• nd. \s in t iwse two examples, practical numerical considerations which 
;m inv! '' Vi " i, huunal mathematics may require alternative methods 

T A ' N ' Wl " t ' (tnir.jdu non to Malf.t >ratk.<, 1 5): ' Mathematics as a 

; : .' " i, -“ -"■•••- puhably «i <... proved propositions about 

17 Ileni.-. without -pe-ifh-ation «.r .Infinite particular thing*.' The 

fmm the detn.N .,f their application in parti- 

nunhi-t^n;- '. " ’ " Kem,rrtUl - V Whi ‘ h ‘••nMdetv.i part of 
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for treating problems for which complete formal solutions may already 

be known. . . . . , 

Another matter in which there is much greater emphasis in numerical 

analysis than in formal analysis is the checking of numerical work. 

Numerical results which are not reliable are of little or no value and tor 

this reason any process for obtaining them should include checking 

procedures for confirming that the alleged results are free from mistakes. 

This is considered further in § 1.3. 

As already mentioned, numerical analysis is concerned with processes 
It is an active subject, one in which things happen in the course of 
carrying out numerical processes, and it cannot be learnt properly simply 
by reading about it. by following examples already worked, or eten b> 
watching examples being worked, any more than one can learn golf, 
tennis, or violin-playing by watching others play, without ever handling 
a club, racket, or violin. There is a great deal of difference between only 
thinking about processes for carrying out numerical calculations and 
actually carrying them out with numbers in the place of the algebraical 
symbols of a general treatment, and the student who wishes to get 
feeling for the subjeet mvst work examples of the processes for himself. 
This is an essential part both of study and research in the subject. Also, 
probably, he must make his own mistakes and spend time finding them 
and correcting them and their consequences before he really appreciates 

the importance of adequate checking. 

The processes of numerical analysis are necessan y Jim t pi s - 

Ideas such as limiting processes, Dcdekind sections, formal convergence 
scarcely play any part in the numerical processes themsc ves, thougl 
they may be involved in the analytical arguments by which the nuinci ical 
process is established. Related to this is the approximate nature of muc i 
of numerical analvsis. In most applications of numerical analysis, almost 
no problems have answers which are rational numbers. But our system 
of representation of numbers is not suitable for numerical operations on 
irrational numbers, so that in most eases wc have to b c satisfied i 
approximations. And even when the answers arc rations numbe e 
shall often be content with decimal approximations to these lational 

numbers, if indeed we would not prefer them. . , 

It will be as well to end this section by explaining what numeral 

an £: i: necessary concerned with the analysis of numbers ol, 

tained by observation in the courseof some branch of experimental science, 

second! V, it is no, closely related to statistics. Certainly numeric al 
% 
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analysis may be involved in the analysis of observational material, 
whether statistical or obtained by measurement, and the analysis of 
observations consisting of measurements may involve consideration of 
the statistics of errors of the measurements. But the subject itself is 
distinct from these two particular applications of it, just as it is distinct 
from its particular applications, for example, to the evaluation of super¬ 
sonic fluid flow or to the structures of atoms or stars. 


1.2. The main types of problems in numerical analysis 

The main kinds of operations which have to be carried out in the course 
of a numerical calculation, and for which numerical processes are re¬ 
quired, are the following: 


(a) Evaluation of formulae. 

(b) Solution of non-linear equations in one unknown. 

(c) Solution of systems of linear simultaneous equations. 

(d) Inversion of matrices. 

(p) Determination of characteristic values and characteristic vectors 


of matrices. 

(/) Solution of systems of non-linear simultaneous equations. 
(<j) Tabulation of standard functions. 

(/<) Interpolation and subtabulation. 

(i) Integration and ditlerentiation of a given function. 

(j) Smoothing. 

(/.•) Integration of ordinary differential equations. 

(/) Integration of partial differential equations. 

(in) Solution of integral equations. 

(-v) Harmonic analysis. 

(o) Frequency analysis (periodogram analysis). 


01 t lies' 1 , (j), (/'). and (o) are often concerned with analysis of observed data. 
which is not primarily the subject of numerical analysis as pointed out 
at the cud of the previous section. 

A single calculation may involve a number of these operations. For 
example, evaluation ^>1 the solution oi an ordinary differential equation 
max well involve any one or more of («), (h), or (//) as well as the integra¬ 
tion process (k) itself. 

I lie subjects in this 1 -♦ have been arranged more or less in order from 

less to more 'advanced ; they will not, however, be taken in this order, 

since some of the ideas required in treating later items of this list are also 
valuable in t lie earlier ones. 
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1.3. Errors, mistakes, and checking 

There are three reasons for which the results of a numerical calculation 
may differ from the exact answer to the mathematical' question con¬ 
cerned : 

(i) One (or more) of the formulae which are evaluated in the course 
of the work is derived by cutting off an infinite series after a finite 
number of terms; the errors introduced in this way are called 


‘truncation errors’; . 

(ii) Only the more significant decimal digits of a number are retained, 

the less significant beyond a certain point being rejected: this 
process is called ‘rounding off' and the errors introduced in this 
way are called ‘rounding errors’ or ‘rounding-off errors ; 

(iii) Mistakes are made in carrying out the sequence of operations 
required to obtain the results sought. 

The distinction made here between an ‘error’ and a ‘mistake’ is this 
A ‘mistake’ is due to fallibility, either human on the part of the mdividua 
carrying out the calculation, or technical on the part of the mechanics 
or electrical aids used in the course of it, and is in principle avoidable. 
‘Errors’, in some degree, are unavoidable, except in some eases of cab 
eolations concerned entirely with integers or rational numbers such 
calculations may occur, for example, in connexion with numb « 
but are otherwise exceptional. ‘Truncation errors are unavoidable in 
any process which takes the place in numerical work of a limiting process 
of Laly sis; integration and differentiation are two important examples. 
‘Rounding errors’ are inevitable in division when the answer is a non 
terminating decimal, and in the use of values of functions other than 
polynomials with rational coefficients, and are often incurred m multiph- 

eatfon also, since although it is possible to retain the (m +") c , g' ts of 
product of two numbers, one of m and the other of » digits, it is only 
exceptionally that all these digits are wanted; if one or both of the 
numbers being multiplied is subject to rounding error, some of the less 

significant digits in the product will be valueless anyway. 

g It is necessary to check that the final results of a calculation are not 
vitiated either by errors or by mistakes, and in a substantial calculation 
it will usually be advisable to include a number of checks of ‘ntermcdia 
results as well. It is often possible to estimate the magnitude ofTru na¬ 
tion errors and so ensure that they are kept below .^specified to emnee 
depending on the calculation and the accuracy required in the final results. 
Rounding errors can often be rendered innocuous by carrying one or tu o, 
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or sometimes more, extra figures, known as ‘guarding figures’, in inter¬ 
mediate stages of the calculation; for example in calculating a compound 
interest table of _ (i-0325) p 

for j) = 0(1)100, to five decimals, by repeated use of the recurrence 
relation /(pH-1) = l*0325/(p), 

rounding errors greater than 6 in the sixth decimal can be avoided by 
keeping eight decimals in the intermediate values of f(p). A full analysis 
of the effect of rounding errors in any but a simple calculation may be 
fairly elaborate. 

Intermediate and final results of a calculation will usually be influenced 
by rounding errors at previous stages of the work, and in some cases the 
accumulated effects of rounding errors will result in checks not being 
satisfied exactly. Let y be the correct value of a quantity and y* the 
calculated value of it. Then there may be a range of values of y—y* 
which can be accepted as being results of rounding (and possibly trunca¬ 
tion) errors and not as indicating mistakes. The term ‘tolerance’ (in the 
sense in which it is used in machining work in engineering) will be used 
for this acceptable range of y—y*. For example, if a check consists of 
the equality of two numbers calculated by different processes, and the 
tolerance of each is -±2 in the last digit, a difference of 3 between them 
in this digit is within the tolerance on this difference, and can be passed. 

Anyone intending to undertake a serious piece of calculation should 
realize t hat adequate checking against mistakes is an essential part of any 
satisfactory numerical process. No one, and no machine, is infallible, and 
it may fairly be said that the ideal to aim at is not to avoid mistakes 
entire!;.. bur to find all mistakes that nn made, and so free the work from 
an\ iitu'lnihjifd mistakes. This of course is an ideal. It does not seem 
possible to eliminate mistakes with absolute certainty; it is always pos- 
dhle that a mistake might be made in the check itself in such a wav as 
to cancel the effect of an error it was devised to find. But with properly 

designed checking procedures and care in working, the probability of this 
should be negligibly small. 

i ro\ Lion of adequate cheeks is not. however, to be regarded as an 
excuse it.r mistakes or a justification oi carelessness in carrying out the 
«h'tails of numerical work. Location and diagnosis of a mistake, and 
correction ot the mistake itself and of subsequent calculations vitiated 
by it. is often a time-consuming job, and a tiresome one at that; and 
moreover, if mistakes are too frequent, the probability of a mistake in 
a check masking a mistake which the check should detect mav become 
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appreciable. Numerical work should always be done with care to avoid 
mistakes, and checks regarded as insurance against the occasional mis¬ 
takes which may occur even in careful work. 

Many calculations consist of the same group of arithmetical operations 
applied repeatedly to different data. For example, if it were required to 
evaluate the function y defined by 
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a: 11 -!-... 


(1.3) 


for a set of values of*, say * = - 3-0(0-1 )3-0, by evaluating and summing 
the separate terms of the series, the process of calculating y is the same 
for each value of * (except that for the smaller values of * more terms 
of the series are negligible and do not have to be evaluated explicitly). 
Such a systematic set of calculat ions is easier to check than one in which 
no step is similar to any other. A single value of the function y would be 
difficult to check adequately; a systematic set of values can be checked 
comparatively easily. In this case, for instance, a check might be based 
on the fact that y defined by (1.3) is a solution of the differential 
equation y" = 1-f xy (for an example, see § 3.3), but use of such a check 
depends on the behaviour of y as a function of x, and is not applicable 
to a single isolated value of y. 

Mistakes in such a calculation are of two kinds, systematic (that is, the 
same mistake is made at the same point in each repetition of the sequence 
of arithmetical operations) and random. These can be illustrated from 
one method of evaluating the above series (1.3). Suppose the (n+l)th 
term is evaluated by multiplying the nth by x 3 /(3n-j-l)(3n + 2); then 

(third term) = (x 3 /56) X (second term), 

and the denominator here might be taken as 54 instead of 56 throughout 
the whole calculation for all values of x; this would be a systematic 
mistake. On the other hand, one too many or one too few zeros between 
the decimal point and the first significant figure might be taken in a single 
one of the terms of the series for a single value of x; this would be a 

random mistake. 

It is recognized by those with extensive experience of numerical work 
that there are two kinds of random mistake which are particularly easy 
to make. One is an interchange of adjacent digits; for example, 28575 
may be read or recorded as 25875. The other is repeating the wrong digit 
in a number in which two adjacent digits are the same; for example 36609 
may be read or written as 33609 or 36009. The error introduced by a 
mistake of the first of these kinds is always a multiple of 9 in terms of the 
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less significant of the two interchanged digits as unit; this may often help 
in locating and identifying a mistake of this kind. These are not, of course, 
the only kinds of mistakes that can be made: but if a check indicates the 
presence of a random mistake, knowledge that it is likely to be of one 

of these kinds may assist in diagnosing it. 

In the calculation of the function y defined by (1.3) by evaluation of 
the series, the calculations for different values of x are independent, so 
that a mistake in the calculation for one value of x does not affect those 

for later values of x. 

But in many calculations, such as a calculation of this same function y 
by numerical integration of the differential equation y = 1 -f -xy satis¬ 
fied by it, a mistake at one stage vitiates all subsequent work. In such 
a case, it is important to have a current check on the work as it progresses 
rather than only an overall check carried out when the calculation is 
completed, otherwise the amount of work that has to be repeated if a 
mistake is made may become very considerable. All the time spent on 
work subsequently found to be vitiated by a mistake is just wasted, and 
a few experiences of this kind may be found severely discouraging, 
although really the moral should be simply that an adequate current 


check is needed. 

One kind of ‘check’ is so inadequate as to be almost worthless, namely, 
repetition of a calculation by the same individual that did it originally. 
It is much too easy to make the same mistake twice; and indeed it may 


be that having made a mistake once, one is conditioned to make it again 
< >n repeat ing t he work. An independent repetition of the work by a second 
individual is better than no check, but should not be regarded as adequate. 
The only really salisfactorv check is one which obtains or verifies a result 

t % 

by a different sequence of arithmetical operations , or a sequence invoicing 
d iffi n nt numbers, from that by which it was obtained. For example, values 
of cosh x and sinh.r interpolated from tables may be checked by use of 
the identity oosh-r—sinh 2 .r — 1 (this does not check that they are not 
both interpolated for the wrong value of x, but this can probably be 
checked in some other way. depending on the rest of the calculation for 
which values of cosh a* and sinh.r are wanted); and the values of y calcu¬ 
lated from the series (1.3) can be checked by use of the differential 
equation satisfied by y. 


1.4. Arrangement of work 

In most numerical work, a working sheet will be used for recording data 
and intermediate results of the calculation. A clear and orderly arrange- 
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ment of this working sheet is a great help both in avoiding mistakes and 
in locating and correcting any that do happen to be made Numerical 
work should not be done on odd scraps of rough paper, but laid out 
systematically and in such a way as to show how the intermediate and 
final results were obtained; and the numbers entered on the work sheet 
should be written neatly and legibly. Use of ruled paper is a he p in 
keeping the layout of the work neat and clear. It is advisable to use loose 
sheets rather than a book since it is rather easy to make mistakes in 
copying from one page to another of a book; with loose sheets the number 
to be copied from one sheet, and the place to which it is to be copied on 
another, can more easily be brought close together, and the copy made 

For work of any permanent value, it is advisable to record on the 
working sheet enough explanation of the different entries, and how they 
were obtained, for the working to be followed after the lapse of a period 

of years. 


1.5. Accuracy and precision 

In contexts in which numerical work is carried out in connexion with 
scientific and technical problems, we are often concerned with the 
numerical solution of one or a set of algebraic, differential, or integral 
equations. Then it may be convenient to distinguish between the 
accuracy to which the equations, or data used in obtaining a solution 
of them, represent the real situation to which they refer, and the accuracy 
to which the results of the numerical work represent the solution of these 
equations with these data, supposed exact. The latter is sometimes 
distinguished by being called the ‘precision' or ‘nominal accuracy of 

the numerical work. . 

Calculations are often carried out deliberately to a nominal accuracy 

known or expected to be higher than the accuracy of the approximations 

made in deriving the equations, or higher than that of the data used in 

their solution. There are several reasons why this may be done. We may 

be interested in the differences between the results of observation and 
of calculation, whether for the purpose of assessing the accuracy to which 
the equations do give an account of the observations, or in order o 
analyse these differences so as to derive more accurate equations or data 
to use in them. Then we want to be sure that the difference between the 
results of observation and of calculation are significant, and are not 
merely consequences of the limited nominal accuracy of the calculations 
Or we may want to determine the difference between two solutions of 
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the equations with different values of some parameters, and to obtain 
this difference we may have to calculate the separate solutions to a 
nominal accuracy higher than that of the data. Or the results may be 
only intermediate results on which some extensive interpolation, perhaps 
in two or three variables, is going to be carried out. Both for the inter¬ 
polation process and for checking purposes, it is then desirable that these 
intermediate results should be smooth and of a nominal accuracy higher 
than required in the final results. 

In a hand calculation, however, greater nominal accuracy means more 
work, more writing in recording intermediate results, more possibilities 
of mistakes, and a longer time for the calculation. It is advisable, there¬ 
fore, to watch lest the calculation is being carried to an unnecessarily 
high nominal accuracy. In this connexion a warning may be given con¬ 
cerning the use of desk machines. Since with a desk machine it is possible 
to work to eight or ten figures, there is a tendency to get into a habit 
of working with eight or ten figures when four or five would be adequate. 
This is bad practice, and a habit which the serious student of numerical 
work should avoid for his own sake. 


II 


THE TOOLS OF NUMERICAL WORK AND HOW 

TO USE THEM 

2.1. The main tools of numerical work 

For carrying out the numerical details of a calculation there are four 
main kinds of tools: 

(а) Desk machines. (0 Slide rule. 

(б) Tables. (<*) Graph paper. 

Of these the first and second are much the most important. 


2.2. Desk machines 

A desk calculating machine is the most important single tool for 
numerical work and anyone intending to study numerical analysis 
seriously should become familiar enough with the main kinds to use 
them with facility, without more deliberate thought for the details of 
operating the machine than a good typist gives to the operation of indi¬ 
vidual keys of the typewriter. 

There are several kinds of desk machine, some being primarily adding 
machines whereas others have facilities for multiplication; the former 
are sometimes called ‘adding machines as distinct iiom calculating 
machines’ to emphasize this feature. The latter are the more important 
and will be considered first; adding machines are considered in §2.26. 
Of the calculating machines some are considerably different m appear¬ 
ance and operation from others, but all arc broadly similar in general 
principle. All have four main components: 

(i) A setting mechanism by which a number can be set on the 

machine. 

(ii) A register in which results of additions, subtractions, and multipli¬ 
cations are accumulated; this will be called the accumulator , 
other names for it are ‘result register’ and ‘product register . 

(iii) A counting register, sometimes called multiplier icgisUi , on 
which a count is kept of the number of additions or subtractions 

made. 

(iv) An operating handle (in hand machines) oi key-opeiate< s\u 
(in electrically-driven machines). 

The setting mechanism and registers have means for setting them to ze ro, 
this is called ‘clearing’. 
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Three kinds of desk calculating machines are illustrated in Figs. 1, 2, 
and 3. Three different kinda of setting mechanisms are represented in 
these three machines, and this is the main reason for the difference in 
appearance between them. 

On the Brunsviga (Fig. 1), the setting mechanism consists of a series 
of levers, one for each digital position, each lever having ten positions 
corresponding to the decimal digits 0-9. On the Marchant (Fig. 2) the 
setting mechanism consists of a keyboard on which there is a set of nine 
keys, corresponding to the digits 1-9, in a column in each digital position, 
a number is set by pressing the appropriate key in each column. On the 
Facit (Fig. 3) there is a keyboard of only ten setting keys, corresponding 
to the digits 0-9; a number is set by pressing these keys in an order 
corresponding to the order of the digits in the number, beginning with 
the most significant. 

The accumulator can be traversed relative to the setting mechanism, 
so that the least significant digital position of the adding mechanism 
corresponds to different digital positions of the accumulator. Shifting 
the accumulator one place to the right corresponds to multiplication 
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On all these machines multiplicat ion is carried out by repeated addition 
and shifting. Machines which carry out multiplication directly, by use 
of a built-in multiplication table, have been constructed, but some 
machines carrying out multiplication by repeated addition are now so 
fast that, for work in which numbers are supplied manually to the 
machine, there is little purpose in making machines using direct 
multiplication. 

On hand-operated machines, addition is carried out by rotating the 
handle through one turn in one direction (clockwise, looking along the 
handle towards the body of the machine) and subtraction by rotating it 
through one turn in the other direction. Most machines have a lock on 
the handle so that once a turn has been started it must be completed, 
and often have mechanical interlocks to prevent incorrect operation. 
On an elect! iealh -driven machine the rotation is supplied by an electric 
motor instead of directly by the operator's hand, and the motor is con¬ 
trolled by a set of kev-oncrated switches. 

* .. 


This is not the place for an account either of the internal mechanism 


or of the details of operation uf different machines; the operating proce¬ 
dure is given in booklets supplied with the machine or obtainable from 
the makers or agents, but can perhaps be best acquired from personal 
demonstration by someone already familiar with the machine. The 
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Fic. 3. Faeit (hand-operated model). 


following sections deal with some general points of procedure applicable 
to most machines. 


2.21. Addition and subtraction 

Addition is carried out by setting the addend on the setting levers or 
keys and turning the handle once positively, or on an electric machine 
by pressing the -f key or by multiplying by 1 with the shift-control set 
to ‘non-shift’; on some machines the latter procedure is necessary when 
it is required to hold the number set, since this is cleared after addition 
when the {- key is used. The number set up is then added to the content 
of tbe accumulator. I lie position ol the decimal point needs watching 
it the number of digits after the decimal point is different in the addend 
and m the content of the accumulator. Decimal-point markers are fur¬ 
nished on all machines (their form is different on different machines); 
m single arithmetical operations it is often unnecessary to use them, but 
they are very useful in helping to keep the position of the decimal point 
correct in carrying out sequences of operations on the machine without 
writing down intermediate results, as is sometimes possible. 

Subtraction is carried out similarly to addition, except that the handle 
is turned m the opposite direction. The result of subtracting a greater 
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number from a smaller is as follows. Let a be written for a contribution 
(—a) in any digital position, so that, for example, the number 90 can be 
written llO and the number 88 as ll2. Then the negative number —23 
(for example) which is 

— 23 = —1,000 + 977 = —10,000 + 9,977 = -1,000,000 + 999.977 

can be written _ 

-23 = 199/ = 19977 = 1999977, etc. (2.1) 

The number 999...99977, to the full capacity of the accumulator of the 
machine, is called the ‘complement’ of 23, or the ‘complementary form’ 
of the number —23; it can be regarded as a representation of the number 
— 23 in the form (2.1), with the digit I to the left of the most significant 
digital position of the accumulator. In a number in complementary form, 
the digits to the right of the row of 9's are the significant figures. 

Negative results appear in such a complementary form, and, in some 
machines, a carry-over from the most significant digital position of the 
accumulator is indicated by the ringing of a bell. 

Recording of negative numbers will usually be in terms of sign and 
modulus, not in their complementary form. The translation from the 
complementary form to the modulus can be done in two ways: 

(i) Translate mentally by subtracting each digit of the complement 
except the last from 9 and subtracting the last from 10. Set the 
result on the setting levers or keys, add into the accumulator, and 
verify that the content of the accumulator is now zero. This checks 
the translation and should always be done before the result is 
recorded. If the number in complementary form is wanted in the 
accumulator for further numbers to be added to it, it can be 
recovered by subtracting the number on the setting levers. 

(ii) Transfer the number in complementary form from the accumulator 
to the setting levers or keys (see §2.22), and subtract from zero. 
This will give some spurious 9’s on the extreme left of the accumu¬ 
lator, but it will be easy to distinguish these from the significant 
figures of the result. 


2.22. Transfer from accumulator to setting keys or levers 

In some calculations it is necessary to transfer to the setting levers or 
keys a number formed in the accumulator as the result of previous calcu¬ 
lations. For example, in the calculation of a continued product, an inter¬ 
mediate product formed in the accumulator has to be transferred to the 
setting levers or keys to be ready for multiplication by the next factor; 
and as already mentioned in the previous section, such a transfer is a step 
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in one method of obtaining the modulus of a negative number expressed 


in complementary form. - 

Some machines are provided with facilities for direct transfer from 

accumulator to setting levers or keys. In using one that is not. the 


following procedure should be followed. 

Copy on to the setting keys or levers the number to be transferred, 

subtract it from the content of the accumulator, and verify that the result ts 
zero. This checks that the number has been copied correctly on to the 
setting mechanism, and this check should always be used.. 

On an electric machine, the subtraction must be done in such a way 


as not to clear the keyboard after subtraction. 


2.23. Multiplication 

Multiplication is carried out by repeated addition in each digital posi¬ 
tion of the multiplier, the accumulator being traversed one place right 
or left between successive digits of the multiplier. In most cases it is best 
to carry out multiplication starting with the most significant digit of the 
multiplier, as then the order of the digits is the natural one, in which it 
is easy to remember the multiplier while the multiplication is being 

carried out. 

In a few r machines, mainly older models, in which the mechanism for 
carrying-over in addition does not extend to the full capacity of the 
accumulator, this procedure wall occasionally lead to incorrect results. 
The best way to test whether a machine has this objectionable feature 
is to subtract 1 from 0 with the accumulator in the extreme left position, 
and see if the carry-over produces 9 s right to the extreme left-hand 
digital positions of the accumulator. If not, the best way of avoiding 
trouble is not to use such a machine; but if none other is available the 
possibility of incorrect results from this cause must be kept in mind. 
In multiplication they can be avoided by starting from the least signifi¬ 
cant digit of the multiplier, but this is inconvenient as it means taking 
the digits in the opposite order to that in which they will naturally be 
remembered. 

Appreciable time can be saved in multiplication on a hand machine 
by a procedure known as ‘short-cutting’. If, as in §2.21, a bar over a 
digit is used to represent a negative digit in that digital position only, we 

have, for example: _ 

183 = 223 (l.h. 12, r.h. 7) 

2879 = 3l2l (l.h. 26, r.h. 7) . (2.2) 

369175 = 431225 (l.h. 31, r.h. 17) 
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Multiplication by one of these numbers can be carried out by using as 
multiplier the number in the form given on the right-hand side of the 
equalities in (2.2), and using both positive and negative directions of 
turning the handle; an appreciable number of turns may be saved in this 
way; this is the process of ‘short-cutting’. The numbersof turns taken to 
carryout a multiplication by each of the numbers in the example (2.2), in 
its form on the left-hand side and in its form on the right-hand side of the 
equality sign, are shown in brackets. ‘Short-cutting’ should be used on 
digits over 5, and on a 5 if flanked on either side by a digit over 5; for 
users of hand machines, it should become the natural way of carrying out 
multiplications; it needs a little practice at first to become proficient and 
reliable, but ease in using it is certainly worth attaining. 

In a few old models of machines, which have not carry-over (sometimes 
called ‘tens-transmission’) mechanism in the multiplier register, only the 
moduli of the individual digits are indicated (in some machines the 
negative digits are indicated in red). Such machines should be avoided, 
or, if they have to be used, short-cutting must be used with discretion 
and particular attention should be paid to checking. 

Some electric machines are fitted with means by which multiplication 
by any digit of the multiplier and the succeeding shift of the accumulator 
can be carried out by pressing one of a set of ten keys; these machines are 
so fast that short-cutting is unnecessary. On others the complete multi¬ 
plier can be set and transferred to a register, then the multiplicand set 
and the multiplication carried out automatically; in these machines the 
operator is not concerned at all with the process of multiplication by 

individual digits. 

2.24. Division 

Division can be carried out in three ways: 

First, by multiplication by the reciprocal of the divisor. This is par¬ 
ticularly useful when the result of the division is required to be in the 
accumulator, either in order to have further numbers added to it or for 
transfer to the setting levers or keys. In the other methods of division, 
the quotient appears in the multiplier register, and no machine has 
transfer facilities from there to the setting levers or keys; this transfer 
has to be done by hand and there is no adequate means of checking it, 
whereas if a result is in the accumulator its transfer to the setting levers 
or keys can be made mechanically or checked (see § 2.22). 

Secondly, by successive subtraction, starting from the most significant 
digit. In this process, the dividend, if not already in the accumulator as 


0303 


c 
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a result of previous operations, is set and added into the accumulator 
which has previously been cleared. The multiplier register must then be 
cleared (this is automatic in the case of automatic division on some 
electrical machines); this is a step which is rather easily overlooked. The 
divisor is then set and subtracted in the most significant position until 
the remainder is less than the divisor; the accumulator is then shifted 
one place left, and the subtraction followed by a shift is repeated. The 
result appears in the multiplier register. In order to make full use of the 
capacity of this register, the divisor should normally be set in such a 
position on the setting levers or keys that the quotient has a non-zero 
digit in the extreme left digital position of the multiplier register. 


b/c Multiplier register 



a 

c 

a b/c j 

b 


Setting levers or keys 
Accumulator 


Fia. 4. (Dots • show decimal-point markers used as separators.) 


Thirdly, by successive addition (sometimes called division by ‘building 
up'). In this process the accumulator is cleared, the divisor set on the 
setting levers or keys, and multiplied by such a number x that the result 
in the accumulator is the dividend. This multiplication is done by a 
process w hich is essentially one of trial, but can be quite fast, and in which 
short cutting can be used to some extent. It is useful when the same 
divisor is used with a number of dividends, as then this divisor can be 
set up once for all and need not be disturbed to set the new dividend. It 
is sometimes also useful for forming able in one operation, if the number 
of digits involved is not too large. If a is set at one side of the setting 
mechanism and c at the other, and bje is found by building up c to 6, 
then the handle has been turned a number of times corresponding to 
(/' r), and a has been multiplied by that number. The arrangement is 
shown diagrammatically in Fig. 4. Another application of the process 
of division by building up is in the calculation of ja/6|, where a is a 
neg.it l\ e number standing in the accumulator (in complementary form) 
as the result of a previous calculation If |6j is set, and the content of the 
accumulator built up to zero, the multiplier register will read j ajb\. 
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Most electric machines are provided with facilities for automatic divi¬ 
sion; the dividend is placed in the accumulator, either by adding it in 
after clearing the accumulator, or by forming it there as a result of 
previous calculations. The divisor is then set and the ‘automatic division ’ 
key pressed; the division then proceeds without further manipulation on 
the part of the operator. 

2.25. Other calculations 

With a machine of sufficient capacity, and numbers of a few digits, it 
is possible to do two calculations simultaneously, one with numbers set 
on the extreme left and the other with numbers set on the extreme right 
of the setting levers or keyboard; an example has already been given in 
the calculation of ab/c in one operation. The following are two other 
examples: 

(i) J t a n and ^a n b n simultaneously. Set 1 on the extreme left and 

n n 

the numbers b n successively on the extreme right; for each b n 
multiply by the corresponding a n . Then in the accumulator 2 (l n 

is formed on the left and 2 a n b n on the right. 

n 

(ii) 2 a l and 2 a n fj n simultaneously. Set a H on the extreme left and 

n n 

b n on the extreme right, multiply by a n , and repeat for each value 
of n. Then in the accumulator 2 is formed on the left and 

H 

2 a n b n on the right. If the multiplier register is not cleared between 

n 

each multiplication, 2 a n is accumulated there, but this is hardly 

n 

satisfactory, as it is then impossible to check after each multiplica¬ 
tion that the right multiplier has been used. An overall check can 
be provided by setting the pairs of numbers a n , b n in succession 
and multiplying each pair by the corresponding b n . This would 
give 2 a n b n anc * 2 bn the latter is likely to be wanted in contexts in 

n n 

which 2 a n an( l 2 °n b n are wanted, and the agreement of the two 

n n 

values of 2 <*„&„ would check that the right multiplier values had 

n 

been used in each calculation; it does not check the setting of tlie 
values of a n in the first, or of b n in the second, of the calculations. 

2.26. Adding machines 

In adding machines the position of the accumulator relative to the 
keyboard is fixed, and there is no multiplier register. Most of them have 
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keyboard setting, and many have electrical operation controlled through 
a set of keys. 

The most useful of these machines are those which make a printed 
record of each number added into the accumulator. There are two 
operations b} r which a total standing in the accumulator can be printed. 
If a key marked ‘total’ is operated, the total is printed and the accumu¬ 
lator is cleared ; if a key marked ‘sub-total’ is operated the total is printed 
and retained in the accumulator. A particular application of the latter 

X 

operation is in the evaluation of an integral j f{w) dw as a function of 

a 

its upper limit x, by successive addition of contributions from successive 
intervals of x. After each contribution is added, a sub-total is taken, then 
the next contribution is set and added. The printed record consists of a 
sequence of entries, alternately contributions to the integral and values 
of the integral itself. The contributions actually used by the machine can 
then be checked against the values which should have been set. 

It should be a convention in using a machine of this kind that it is left 
with the accumulator clear; but in case this has not been done, it is 
advisable, before using it, always to ensure that the accumulator is clear 
by taking a total. 


2.3. Mathematical tables 

Mathematical tables form a very important aid to numerical work. 
Many calculations involve the use of values of standard functions such 
as exponentials, logarithms, circular functions, Bessel functions, the 
gamma function, and though it would be possible to calculate the required 
function values from scratch as they were wanted, this would usually 
lengthen the calculation so much as to make it impracticable. In fact, if 
tables of these functions did not already exist, it would often be worth 
constructing them as a first step in the calculations for which values of 
these functions are wanted. 

The most important tables are the following: 

Comrie and Milne-Thomson's Standard 4-Figure Tables ; Chambers's 
6-Figure Mathematical Tables (1948-9), edited by Comrie; Barlow's 
1 abics of Squares, Cubes , Reciprocals, etc., edited by Comrie; Inter¬ 
polation and Allied Tables (H.M. Stationery Office). 

The first two of these include tables of circular functions for argument 
m radians, and also tables of inverse trigonometric and hyperbolic func- 
u..ns ; ot the two volumes of Chambers’s 6-figure tables, the second with 
so-callci 1 nut ural values is much the m ore useful for work with machines. 
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Interpolation and Allied. Tables contains a great deal of information on 
formulae and methods for interpolation and other numerical processes 
and is a very useful and inexpensive booklet. 

For functions other than the elementary functions, the following are 

useful: 

Dale, 5-Figure Tables of Mathematical Functions] ; Jahnke-Emde, 
Tables of Functions with Formulae and Graphs ; British Association 
Tables of Bessel and Airy Functions. 

The amount of tabular material available in various volumes of tables 
and scattered among various journals is very considerable. The nature 
and location of most of this material published up to the end of 1944 has 
been classified and tabulated in an Index of Mathematical Tables f w liich 
is a most valuable volume and should be known to all undertaking any 
extensive numerical work, or even small calculations involving functions 
other than the elementary functions, since if a function has been tabu¬ 
lated, knowledge of this fact and an adequate reference will usually avoid 
duplicating the calculation of it. 

An important source of information, particularly regarding recent or 
current work on tabulation of functions, is the journal Mathematical 
Tables and Aids to Computation (generally referred to as M.T.A.C.). 

In describing a table, it is convenient to have a compact notation lor 
specifying the range and interval of the argument. The notation 
x = x x {hx)x 2 has come to be adopted as the standard abbreviated form 
for ‘for values of a: from x l to x 2 inclusive at intervals 8x\ The notation 
ml) or nS is often used for a table to m decimals, or n significant figures. 

2.31. Critical tables 

Most tables give the values of the function/(x), rounded off to a certain 
number of decimals, for a sequence of equally spaced exact \ allies of the 
argument x. Occasionally another type of table is more convenient, 
namely, one giving the range of x for which the function/(x), rounded oil 
to a certain number of decimals, has a specified value. Such a table is 
called a critical table, and is convenient for slowly varying functions, and 
also for functions which have a limited range and for which accuracy in 
the last figure is important; in using a critical table no interpolation is 
required, and the possibility of an error of a unit in the last figure in 
interpolation in an ordinary table is avoided. 

As an example, consider a table of $x(x—i)(x— 1). which occurs in a 

t By A. Fletcher, J.C. P. Miller, and L. Rosenhead (Scientific Computing Service, Ltd., 
London 1940). A second edition is in preparation. 
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formula for non-linear interpolation, as a function of x. A portion of a 
critical table of this function to four decimals is as follows: 



X 


0-1621 

+ 0-0077 

0-1691 

0-0078 

0-1777 

0-0079 

01897 

0-0080 

0-2334 

0 0079 

0-2462 



The values of f(x) are on lines intermediate between those on which the 

values of x stand, and the values of x between which a value of f(x) stands 

mark the limits of the range of x for which that is the rounded value of/(a:). 

These values of x are rounded values of the inverse function f~ l (y) for 

values of y = /(.r) halfway between the tabular values. For example, the 

above table indicates that for values of a: between 0-1777 and 0-1897 the 

• 

function/(.c) has the value -f-0-0079 to four decimals; and the value of x 
for/(.r) = 0-00785 is 0-1777 to four decimals. 

It is a convention in critical tables that if x has exactly the tabular 
■\ alue, t he value ot f(x) to lie taken is that standing above the line on which 
f(x) stands; a reminder of this convention is often given in such tables 
by the words ‘in critical cases ascend’. 


V variables in tables 

An important aspect of mathematical tables is the use of auxiliary 
variables to simplify interpolation. This is especially important (i) in the 
neighbourhood of a singularity, where the ordinary interpolation formu¬ 
lae. applied directly to the function values, cease to be valid, (ii) for large 
values of the argument, and (iii) for oscillating functions when the table 

has to cover a large number of periods of the oscillation. Use of auxiliary 

variables may both simplify interpolation and lessen the amount of 
nmtmal winch has to be calculated and printed to provide a useful table. 

I hv nm,t usual step is to tabulate an auxiliary function, but in some 
va-< s an auxiliary independent variable may be used instead or in addi¬ 
tion. « lie following are examples of the tabulation of auxiliary functions: 

0) io g(K'» z),'x) and log|(tan x) '.rj for small a*, in place of logsin a: and 
logianr vInch are infinite at .r = 0 and cannot be interpolated by 
standard lormulae near a; — 0 
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(ii) If f(x) is oscillatory, it may be possible to determine an ‘amplitude 
function’ A(x) and a ‘phase function’ <j)(x) such that 

f(x) = A(a;)cos{<£(:r)} 

and that A (a?) and <f>'(x) vary much more slowly than /(.r). Then 
A(x) and <f>[x) can be tabulated at wider intervals than f(x), and 
interpolation of A (x) and cf)(x) is easier than that of f(x) itself. This 
is particularly convenient when two functions can be expressed 
as the real and imaginary parts of A(x)exp{i(f>(x)}. An important 
example is provided by the Bessel functions, for which 

A(x) = x>[{J n (x)Y+{Y n (; r)} 2 ]* = xi\H«>(x)\ 

and (f>{x) = tan - l [Y n (x)/J n {x)] = arg[tf ( n l) (a:)] 

form a convenient pair of auxiliary functions except for small 
values of x. 

An example of the joint use of auxiliary functions and an auxiliary 
independent variable is provided by the elliptic integral 

7T 

K(k) = | (1 —fc 2 sin 2 0) -i dd 
0 

near A: =1. lik' = (1 —£ 2 )* and 

K(k) = A r 1 log(4/£')-f-A r 2 , 

K l and A 2 are regular functions of k' near k = 1, and a convenient tabula¬ 
tion is K l and K 2 against k' as argument. 

2.4. Slide rule 

A slide rule is an instrument of limited accuracy, and of limited scope 
since it cannot easily be used for addition and subtraction; but within 
its limitations it is a valuable tool of numerical work. Two contexts in 
which it is particularly useful are the following: 

(i) When a function is tabulated at intervals too large for linear inter¬ 
polation between tabular values, more elaborate interpolation 
formula have to be used; these will be considered in Chapter V. 
In many of these, the interpolated value is expressed as the sum 
of the value which would be obtained by linear interpolation, and 
some other terms which can be regarded as contribution to a 
‘correction’ to this value. For some or all of these contributions, 
the accuracy attainable with a slide-rule may be adequate, and 
then it is a useful tool. 

(ii) When in the solution of a linear differential equation a particular 
integral /'and a complementary function C have been evaluated, 
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and a small constant multiple of C, say yC, has to be added to P 
to give a solution satisfying specified conditions, the calculation 
of y C may often be carried out to adequate accuracy on a slide-rule. 
This is a quick calculation, because after a single setting of y, all 
the values of yC can be read off without resetting. 

As well as the usual straight slide-rule with a 10-inch scale, there is 
another form, with two cursors and a single scale in the form of a helix 
on a movable cylinder. In the Fuller slide-rule this scale is 50 feet long, 
and this enables an accuracy of 1 in 10,000 to be obtained without diffi¬ 
culty, and 1 in 20,000 with care. Such an instrument is cheap compared 
with a desk machine and may be found very useful in work for which its 
accuracy is adequate and in circumstances in which the cost of a desk 
machine is prohibitive. With one of these slide-rules and an adding 
machine much useful numerical work can be done, especialty in contexts 
involving empirical or experimentally determined functions not specified 
to more than four- or five-figure accuracy. 


2.5. Graph paper 

Graph paper is more generally useful as a means of presenting results 
than as a tool tor obtaining them. But there are occasions when it is 
useful as a means of doing calculations, e.g. for obtaining approximate 
results which can later be refined bv more accurate methods. 

Before being used for anything more than qualitative or the roughest 
of quantitative work, graph paper should be examined for uniformity of 
ruling. Paper ruled in two colours (e.g. blue for the main ruling, with 
red toi every tenth line) should be examined for the registration of the 
two colours. Paper which is ruled with every fifth or tenth line thick 
should be examined to see that the intervals between the centres of the 
lines are umlorm, and not the intervals between the edges of the lines, 
a remarkable fault in some papers.f 

2.6. Other machines 

l here are other aids to numerical work of various kinds, but mostly 

large or special pieces of equipment which are unlikelv to be available 

% 

to most of those for whom this book is primarily intended. The more 
important may, however, be mentioned here. 

1' irst. there is the .National machine.* developed from an accounting 
machine. I his is an adding machine with keyboard setting mechanism 

[ S.v .U ft jo vs iin.i J off rt\vs. Methods of Mathematical Physics, rhnp. 9. 

♦ b. J. Ounrio, Jour,,. Iioy. Sint. Soc., Supplement, 3 (1936), 87. 
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and six registers, with facilities for adding or subtracting the number set 
on the keyboard, or the number standing in any register, into any 
combination of registers. The mechanical arrangement for controlling 
these transfers is such that it can only be used effectively in calculations 
in which the same set of operations has to be repeated successively on 
different sets of numbers; but many calculations have just this character, 
and for such calculations this machine can be very valuable. 

Secondly, there are two groups of machines for carrying out arith¬ 
metical operations on numbers represented by punchings on cards, the 
‘Hollerith’ and ‘Powers-Samas’ machines. The main machines of each 
group are a ‘tabulator’ which is a multi-register adding machine with 
printing mechanism, a sorter, and a multiplying punch which can take 
a card with two numbers punched on it, and calculate and punch their 
product. The use of these machines, and the organization of calculations 
for them, is a special technique of its own,| and hardly appropriate for 

an introductory book like the present. 

Thirdly, there are various high-speed automatic calculating machines 
which can carry out, automatically, long sequences of operations once 
they have been supplied with operating instructions in a suitably coded 
form. A short account of the principles of these machines and of the 
process of organizing calculations for them is given in Chapter XII. 

2.7. The organization of a calculation 

The organization of a calculation is represented diagrammatically in 
Fig. 5. There are three kinds of equipment the computer has to assist 
him; these are represented by rectangular blocks in the figure. One is 
a desk machine, another is a set of tables, and the third is the working 
sheet on which intermediate and final results will be recorded and on 
which should be written enough data to identify the calculation and to 

summarize the calculating procedure. 

When the method for doing a calculation has been decided, the det ailed 
process of carrying it out consists of (i) a sequence of arithmetical opera¬ 
tions carried out on the machine, or perhaps on a slide-rule as auxiliary 
equipment, or mentally in the case of simple operat ions such as multipli¬ 
cation or division by 2 or addition of pairs of numbers, and (ii) transfer 
of numbers between the three blocks represented in Fig. 5. For example, 
suppose that in the course of a calculation it is required to obtain the 
value of y sin ax where a is a constant whose value has been entered in 
the work sheet as part of the data of the calculation, and x and y are 

t Soo W. J. Eckort, Punched Card Methods in Scientific Computation (Columbia 
University, Hi 10). 
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numbers which have been calculated as intermediate results in the course 
of the calculation and noted on the working sheet. First x must be 
transferred to the machine and multiplied by a. The value of ax will 
probably be transferred to the working sheet for reference; this will be 
necessary if the machine is going to be used for interpolation of the value 


Tables 



Fig. 5. 


of sin(a;r), since the value of ax in the accumulator will be lost in the 
process. 1 lie value of ax will certainly be used as argument in a table of 
sines, and if interpolation in this table is involved, a subsidiary calcula- 
lion is necessary for that purpose. The value of sin(ax) will probably be 
recorded on the working sheet, and certainly transferred to the machine 
lor the multiplication required to form ysinr/a*. 

I Iv transfers and the arithmetical operations are controlled by the 
individual who is carrying out the calculation, who is represented by the 
large circle m Fig. 5; the controls he exerts are represented bv directed 
ne-s Iron, the controller to small circles representing control of the 
'ransfer of numbers, and to the desk machine. He also takes from the 
working sneet information about the arithmetical operations to be 

f a ," ic ''l lienci "' "lis is represented by the directed line 
loan ttic vork sheet to the controller. 
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3.1. The significance of formulae in numerical work 

The evaluation of a given formula is the simplest kind of problem 
in numerical analysis. In a sense most problems reduce to this, as the 
numerical work itself almost always consists in substituting particular 
numerical values into a process or sequence of operations which could 
be expressed in the form of a sequence of formulae to be evaluated, even 
if they are not explicitly so expressed. In most cases the real question 
of numerical analysis is, What is the best formula or set of formulae to 
evaluate in order to obtain the required result?, and it is with this ques¬ 
tion that we shall primarily be concerned in later chapters. But equally 
important questions for practical work are how to evaluate the formulae 
and how to check the results. 

A formula for a calculation to be carried out numerically has a 
significance rather different from that of a formula in formal algebra or 

analysis. For example, the formula 

y = (x*+\)l2x (3.1) 

regarded as an algebraical formula states a relation between the quanti¬ 
ties on the two sides of the sign of equality, and is completely equi- 

vakmtto **-2yx+l = 0 

or x = y±{y 2 — 1)*. (•*--) 

which are different ways of expressing the same relation. But formula 
(3.1) regarded as a formula for a numerical calculation specifies a 
process to be carried out for determining the value of y given the value 
of x, whereas formula (3.2) specifies a process to be carried out for deter¬ 
mining the value of x given the value of y. These processes are different 
from one another, the data used in them arc different and the results 
required are different. This aspect of a formula, as represent ing a process 
consisting of a set of operations to be carried out in a definite sequence, 
plays little part in formal analysis, but is fundamental in numerical 
work. Even the formulae 

x = y—z and 2 = y—x 

mean quite different things when regarded as specifications of numerical 
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calculations to be carried out; and the process specified by the formula 

y = #*+(!/*)] (3.3) 

is different from that specified by formula (3.1). 

A striking example is discussed in § 11.3, where it is shown that of 
two ways of writing the recurrence relation for the Bessel functions, 

namely ’ Jn+ l(*) = (2# l*)J n {x)-Jn-l(*) 

and J n {x) = (xl2n)[J n+l {x)+J n ^(x)] 


which are formalty completely equivalent (for n > 0, x > 0), the first 
specifies a numerical process which is quite impracticable as a general 
method for calculating J n (x ) for n > x > 0, whereas the second gives a 
quite practicable iterative process. 

There may be various ways of evaluating even simple formulae, and 
the best way may depend on the equipment available for carrying out 
the numerical work. For example, in the evaluation of ( abc ...)/(uvw ...) 
by means of a slide rule it is best to take multiplications and divisions 
alternately, expressed by writing this fraction in the form 


[{{a/u)xb}lv]xc.... 

But with a desk machine it is best first to evaluate the denominator 
D — uvu;... and record this, then form the continued product abc..., and 
finally divide the result by I). In forming these continued products, no 
intermediate results need be written down; the only numbers to be 
recorded are 1) and the final result. 

In using a machine it is worth while planning the calculation in such a 
way that as much as possible of the work is done on the machine without 
recording intermediate results, so as to reduce the amount of writing, with 
the possibilities of mistakes in recording and reading the written results, 
to a minimum demanded by the need for clarity in presentation of the 
calculation and for checking. Transfers from the counting register to 
the setting levers or keys should also be avoided if possible. 

Bor example, iiV were given, 2(eosh.r-l) could be calculated from 

2(coshx—1) = e x -f (l/e- r )—2; 

this would require a reciprocal to be calculated, recorded, and reset on 
the machine (or at least transferred from the counting register to the 
setting levers or keys). But if it is calculated from 


2(cosh.r— 1) = (^— 1 ) 2 /^ 

this can all be done by a sequence of operations on the machine alone; 
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it also has the advantage that for small x it does not calculate the result 
as the small difference of two relatively large quantities. 


3.2. Evaluation of polynomials 

Expressions consisting of a number of additions and multiplications 
can usually be evaluated in various ways, of which the best to use in 
any case may depend on particular features of that case. For example, 

a polynomial y _ aoX n+ ai x n-i + ...-fa n _ x .r-fa.. (3.4) 

may be evaluated by calculating the separate terms and adding. When 
x has a simple numerical value {r = 1, 2, or 10 for example) this may be 
the best method, especially if the coefficients are small integers. If this 
method is used for evaluating a polynomial both for positive and for 
negative values of x, a convenient procedure is first to sum separately 
all the terms involving odd powers of x and all those involving even 
powers of x, for positive values of x only, then for each value of x to add 
and subtract these two sums. 

If x has not a simple numerical value it may be better to write 


V = [{(a 0 *+ a i)*+ a 2) a; + a 3]- i: +.-. ( 3 - 5 ) 

and carry out an addition and a multiplication alternately as indicated 
by this expression. That is, construct the sequence y i defined by 

Vq — a o> yi = Vj-i x + a j 0 *>°); ( 3 -°) 

the result required is y n . This process requires n multiplications and n 
additions, and no recording of intermediate results. Care is necessary 
with the decimal point ; use of the decimal point markers is a great help 
here. 

The process for checking the results will depend on the calculation 
of which the evaluation of the polynomial forms part. It is unlikely that 
just a single value of a polynomial will be wanted; the evaluation of the 
polynomial is much more likely to form part of a larger calculation, which 
may well include means of checking the value obtained for the poly¬ 
nomial. 

If a set of values y of a polynomial (3.4) for a set of values of .i is cal¬ 


culated, then 


= aoI*'‘+ a i2 z ' H 1 +- 


(3.7) 


where the sum is over all values of x for which the polynomial has been 
calculated. One way of checking such a set of values of y is to evaluate 
the right hand side of (3.7) and compare the result with 2 y > the results 
should not differ by more than the tolerance for rounding errors. 
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We shall see later (§4.42) that if a polynomial, has simple coefficients 
and is of not too high order, its values for a set of equally-spaced values 
of x can be obtained simply and conveniently by a sequence of additions, 
without any multiplication at all. 

3.3. Evaluation of power series 
To evaluate the sum of a power series 

V — «o+ a i a; + a 2 a:2 4--** (3.8) 

it is often most convenient to write each term as a multiple of the 
preceding one, thus: 

and to evaluate each term from the previous one by the appropriate 
multiplications. Series containing odd powers only or even powers only 
can be treated similarly. If several values of y, at equal intervals of 
x, are calculated, evaluation of the finite differences (§4.2) of the values 
of y probably provides the best check. 


Example: To evaluate 


• v - ! ' !+ r!i l!+ 2Xiv I,+ 


to six decimals for a* ~ 1 *0(0*1 )1 -4. 
If is convenient to write this 


1 


8 *2.4.5.7.8.10.11 

• • . * 


x» -f... 


>J —■ V,-{ I -l—— x 3 4---?-a -9 4-1_x 9 4- 1 

- L 4.5 r 4.5.7.8 1 4.5.7.8.10.11* + 


(3.9) 


and first to sum the series in the square bracket and then multiply the sum by |x 2 . 
It the ratio* of successive coefficients in the series are written b . then 


(*ith term) - &„.r 3 [(n- l)th term]; 
the s aims of the lust few b'a nro 


(3.10) 


6 1 = -1 - 1 
i.5 20 


and in general 


- 7 J s - s- ». - 

6 , __ L_ 

(3n 4-1 )(3»j -j-2) ’ 


1 


1 


10.11 110 


t •••» 


1 ho denominator in this fraction is a quadratic function of m, hence the second 

dil or, nees (sen !; 4.2) of its values arc constant, and this can be used to check these 
values: 

20 f,0 110 182 272 380 

54 72 90 IDS 

I* 18 IS IS 

A similar cluck can usually be applied if the 


ratios of successive coefficients can be 
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expressed as the ratio of the two polynomials of low degree in n. The work can 
conveniently be arranged as follows: 


X 

s 3 

I 

1-0 

1-000 

1-1 

1-331 

1-2 

1-728 

13 

2-197 

14 

2-744 

b n 

1-000000,00 

1-000000,00 

1-000000,00 

1-000000,00 

1-000000,00 

1/20 = 05 

0-050000,00 

0 066550.00 

0-086400,00 

0-109850,00 

0-137200,00 

1/56 = -01785714 

892,86 

1581,75 

2666,06 

4309,65 

6722,80 

1/110 = 00909091 

8,12 

19,14 

41,88 

86,08 

167,70 

1/182 = -005495 

0,04 

0,14 

0,40 

1,04 

2,53 

1/272 = -003676, 

sum . 

0,00 

1-050901,02 

0,00 

1-068151,03 

0,00 

1-089108,34 

0,01 

1-114246,78 

0,03 

1-144093,06 

. 

y (to six decimals) 
b*y . 

0-5 

0-525451 

0-605 

0-646231 

17147 

0-72 

0-784158 

19454 

0-845 

0-941539 

22291 

0-98 

1-121211 

t/* = 1 +xy . . 1-52545 

sy . 

5 y . 

y-ibsy 

1-71085 

4474 

1-7145, 

1-94099 

5287 

169 

1-9453, 

2-22400 

6269 

2-2292 a 

2-56970 


Notes: (i) The entries in the third to eighth lines uro the values of the terms in the square 
bracket in formula (3.9). Each is calculated from the preceding one by formula (3.10); 
if the decimal values of b n given on the left aro used, these terms can bo calculated 
entirely by multiplication and transfer. 

(li) To obtain six decimals in the final result, it is advisable to keep eight decimals in 
tho individual terms, that is, to retain two guarding figures. 

(iii) Tho function y dofined by the series (3.9) satisfies the equation y‘ = \+xy. Tho 
second differences (see § 4.2) of y can be calculated from tho values of y (see § 4.43) and 
compared with tho values calculated from y’ by formula (4.19); this provides a close 
check on tho results. 

3.4. Kinds of formulae to avoid 

There are two kinds of formulae to be avoided if possible, namely 
those that express the result required as 

(i) the ratio of two small numbers, 

(ii) the difference of two large, nearly equal, numbers. 

When one or other of these situations occurs, it often, though not 
always, means that the method adopted for calculating the result is not 
the most suitable, and it is usually worth examining whether there is a 

more suitable alternative. 

The following are some examples: 

(a) Exponential extrapolation 

Three numbers y 0 , y v and y 2 are known to differ from the required 
result Y by amounts which are in geometrical progression; to find 1 (sec 
Fig. 6). This process is called ‘exponential extrapolation’; it is useful in 
some methods of successive approximation (see §9.32). 
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Since y 0 —Y, y x —Y , and y 2 —Y are in geometrical progression, 

(Vz-m*h-T) = (yi -Y)Hy 0 -Y) 

and solution for Y gives 

Y = (2/o 2 / 2 —2/f )/( 2/2 — 2 2/i+ 2/o) * ( 3 - n ) 

But if 2/0 = 2 /!== 2 / 2 .this gives Y = 0/0 which is useless for numerical 
work; and if y 0 , y x , and y 2 are only slightly different from Y, it gives Y 
as the ratio of two small numbers, the numerator and denominator 
being both of order (y 0 — Y). 

But if Y is written as the best approximation y 2 plus a correction, 

thus: Y = y 2 -{y 2 -y 1 ) t Hy2-^i+yo). ( 3 - 12 ) 

the numerator of the ‘correction’ is of order ( y 0 —Y ) 2 whereas the 
denominator is of order (y 0 — -Y)\ the correction is therefore of order 
(y 0 — 7) and is zero in the case y 0 = y x = y 2 > and is small if y 0 is nearly 
equal to Y. Its evaluation gives no trouble. 

This illustrates the way in which two expressions, formally equivalent, 
may be very different when assessed from the point of view of the ease 
of practical numerical evaluation. 



Fio. 0. 


{b) Solution of a quadratic equation when the ratio of the roots is large 
Let x x be the larger and .r 2 the smaller of the roots of the equation 

x 2 -18x+l = 0. 

Use of the standard general formula for the root gives 

x v x 2 = 9±V80. (3.13) 

If V80 is taken to four decimals (five figures) tins gives 

x x , x, = 17-9443, 0-0557. 

Here r 2 is obtained as t he small difference of two relatively large numbers 
9 and \80 = 8-9443; the lirst two significant figures in the value of V80 
are lost, and from a live-figure value only a three-figure result is obtained. 

On the other hand if x 2 is obtained not from (3.13) but from the 
relation (tor this equation) x 2 = 1, the value of ,r 2 is obtained to full 
five-figure accuracy without requiring that v'80 should be obtained to 
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any greater accuracy than for x v Here again we see a marked difference, 
from the point of view of numerical evaluation, between two formally 
equivalent formulae. 


3.5. Evaluation of a function in the neighbourhood of a value of 
the argument at which it becomes indeterminate 
In the neighbourhood of a value of the argument at which a function 
becomes indeterminate, some form of series expansion will usually be 
available. 

Consider, for example, the function y defined by 

ly = (l/sinx) —(1/ar) (0 < |z| < tt) 

ly(0) = 0. 

To evaluate this for small values of x, it is convenient to write it 



x—sin x /sin x 


x z 



x 4 

4 . 5 . 6.7 



2 . 3 . 4.5 



It would be possible to carry out the division of one series by the other 
algebraically, but if more than the first two or three terms have to be 
included, it is easier to evaluate the two series separately and carry out 
the division numerically. 


6363 


D 
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FINITE DIFFERENCES 


4.1. Functions of a continuous variable in numerical analysis 
In numerical work we may be concerned with two different ways of 
specifying functions of a continuous variable. First, a function may be 
specified by a formula which can, in principle, be evaluated for any value 
of x as required: examples of such functions are polynomials, circular, 
exponential, and other functions defined or expressed in terms of conver¬ 
gent power series, and functions defined by definite integrals such as 
the gamma-function m 

T( x) = j e-*F-'dl. (4.1) 

0 


Secondly, there are those functions which are specified only by tables of 
values; these may often be tables expressing some empirical physical 
relationship, such as the relation between grid voltage and anode current 
in an electronic valve, or between velocity and resistance for a projectile; 
or they may be results of previous calculations. 

In practice, there is not much difference between functions specified 
in these two ways, for usually one obtains values of functions of the first 
kind from tables rather than by evaluating the defining formulae. In 
fact, mathematical tables are made precisely for the purpose of enabling 
function values to be determined without going back to first principles 
and evaluating the defining formulae each time a function value is 
required; if we require r(l-27836) we interpolate in tables of r(x), 
rather than evaluate the integral in formula (4.1) for x= 1-27836 
unless it happens that no tables to the number of figures required are 

? ' V aih t bl f; Thus 111 either case we are concerned in practice with functions 
spccihed by tables, and with the properties of functions so specified. 

A function/(a-) specified only at discrete tabular values of the inde¬ 
pendent variable * is not formally defined for intermediate values. If 
ie tabular values of * include zero and are at equal intervals 8x, and 
tf -0 18 any function (not necessarily even continuous) which is finite 
! S0 /then f(x)+g(x)sin(irxj8x) has the same values 

ti n lC Va,UeS ° f Further > the tabular values of/(*) are 

•u m it | Tfi r0Unding eiT01 ' S ’ 80 that the function may not be 

accurately defined even at the tabular values of* 

On the other hand, a table of a function of a continuous variable * 



FINITE DIFFERENCES 


35 


would often be of little value unless it were possible to determine values 
of the function for values of x between the tabular values (to an approxi¬ 
mation depending, of course, on rounding errors). In order to do this, 
some understanding is necessary about the behaviour of the function 
between its tabular values, an understanding which may be justified 
formally in cases of functions of the first kind mentioned at the beginning 
of this section, but may have to remain an assumption in the case of 
empirical functions. This understanding may be expressed qualitatively 
by saying that the function is ‘smooth’ over the range concerned. 
‘Smoothness’ of a function is a property which it is difficult to define in a 
quantitative way; it is discussed further in § 11.4. It implies differentia¬ 
bility to some high order, and smallness of high-order derivatives. An 
example will illustrate this. 


We shall later (Chapter V) derive interpolation formulae for use when the interval 
of tabulation is too large for linear interpolation between tabular values of the 
function. It will be found that it is possible to interpolate sin z, not only roughly 
but to any required accuracy, from its values at interval 8z = ±n: 


— \tt — 7 T 

1 0 


-\tt 
- 1 


Z 

y = sinz 

or even from its values at intervals x = In: 


0 

0 


+ %n 

1 


n 

0 


in 
- 1 


(4.2) 


— 2rr — 3 ~n ~\n 


0 

0 


4 ^ 

4V3 


$n 


-iv'3 


2n 

0 


x 

y — sin x 0 £\'3 — £>3 

Let us inquire what particular property the function y = sinz has which selects 
it from all other functions with the tabular values (4.2) as the one for winch the 
interpolation formula is accurate. Suppose, for simplicity, that we know that y is 
an odd function of z, periodic with period 2 n. Then it can be expanded in a sine 
series in the interval — n < z < n: 

y = a,sinz +a*sin 2x + a 3 sin 3z + ... (4 3) 

where, to give the value of y at z = \n 

1 = a l — a 3 -fa 4 — + . (4.4) 

We filial 1 require a measure of the nth derivative of y; this derivative varies with j , 
but a convenient single quantity giving an overall measure of its magnitude is its 

mean square value „ 

_L f (,,<">)= t/z = A(a? + 2 n «2'4- 3"<i5 + ...). 

2zr ' 


J 
— n 


As n -> oo. the minimum value of this quantity, subject to the condition (4.4), is 
given by a, = 1, o m = 0 (m > 1). Thus the relevant property ot the function 
y = si,/z is this, that of all functions which are odd and have period 2 tt, ,t «s t ho 
one for which, to put it roughly, the high-order derivatives are us small as possible. 

In the great majority of cases, functions are tabulated at equal 
intervals of the independent variable, which is often called, in this 
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context, the ‘argument’ of the table. For the present, we will only 
consider such sets of function values. 

4.2. Finite differences 

The most important property of a function specified by a table con¬ 
sists of what are called its ‘finite differences’. The following example 
illustrates what is meant by this term: 




First 

Second 

Third 

X 

f{x) = l/x 

differences 

differences 

differences 

30 

•33333 

-1075 



31 

•32258 

-1008 

67 

-6 

3-2 

•31250 

- 947 

61 

-5 

3-3 

•30303 

% 

56 

-6 



- 891 


3-4 

•29412 

- 841 

50 

-2 

3 5 

•28571 

- 793 

48 


3-6 

•27778 





The ‘first differences’ are obtained by subtracting each function value 
from that for the next greater tabular value of x; the ‘second differences’ 
are obtained by carrying out a similar set of subtractions on the first 
differences, and so on. 

Values of odd order differences should be written on levels intermediate 
between those of function values, and of even order differences on the 
same lines as function values, and normally these values are written in 
terms of the last digital position as unit, decimal points, and zeros before 
the first significant digit, being omitted. They can conveniently be dis¬ 
tinguished from function values by being written or printed smaller. 

The finite differences of tabular functions play a very important part 
both in the analytical and in the numerical manipulation of such func¬ 
tions. Use of them enables formulae for operations on such functions, 
such as interpolation and integration, to be expressed compactly and in 
a form convenie.it for practical use. When the tabular values provide all 
the information we have about a function, all processes involving this 
function have to be expressed as operations on the tabular values; one 
of the most important operations on a set of values at equal intervals of 
the independent variable is that of differencing, and we shall later in this 
( napter, and in C hapters \ and \ I, see that most of the other operations 
can be expressed in terms of this one. 
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It will be seen that the values of the third differences in the above table 
are noticeably irregular; this is an effect of rounding errors in the function 

value, which will be considered more fully in § 4,44. 

In order that a function shall be well determined by a table, the 
average value of the nth order differences should tend to zero, or at 
least become small, as n increases. We have seen examples in which this 
does not appear to be the case; for the function y = sinx, at interval 
u in * (see (4.2)), the 2nth differences have extreme values ±2", but 
this function is still weU defined by these values, in the sense that accurate 
intermediate values can be interpolated between them But this is a 
peculiar property of the function y = sin z alone out of all the functions 
with these tabular values; given these function values alone, without the 
knowledge that they are intended to represent sin x, one could not be at 
all confident about the results of any attempt to interpolate between 
them. Eight values per cycle is about the smallest number which can in 
practice be regarded as specifying an oscillating function adequately, 
and at least twelve values per cycle is preferable. 


4.21. Notation for finite differences 

Let x 0 be one of the tabular values of x, x, = (x 0 +j8x) a set of other 
tabular values, and f, = /(*,) the values of f(x) at the tabular values of x. 

There arc two kinds of notation for finite differences. In one the 
differences of a function / are written 8/ or A/, so that the symbol 8 or A 
stands for an operation carried out on the values of the function/. In the 
other the symbol 8 or A is used for the differences themselves. 

The former seems much the preferable, both for use in the derivation 
and manipulation of formulae in finite differences and in application of 
them. It is more nearly self-explanatory, and many formulae with which 

wc shall be concerned express relations between differences of two different 

functions (for example a function of x and its derivative), and if a symbol 
is used to represent a difference itself rather than a difference-operator, 
differences of different functions cannot be distinguished except y 
introducing new symbols, which are unnecessary. In tins notation 
repetition of an operation is expressed by the use of an index (as in 8 , ). 

The use of the symbol 8 or A for the differences themselves ,s a con¬ 
venient shorthand in cases in which it is unambiguous, and ,s sometimes 
preferred by those carrying out the details of the numenca1 work In 
this notation the use of dashes (A') or Roman superiors (as A< ) is prefer¬ 
able to the use of numerical indices to indicate orders of differences. 

In this book, the former usage will be adopted throughout, so that 8 
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and A must be regarded as finite-difference operators. Consistently with 
this notation, Bx will be used for the interval in the independent variable 

(other notations for this are h and ?e).f 

The first difference/!—/„ may be associated with the argument value 
x Q , with the argument value x lf or symmetrically with these two argu¬ 
ment values, and assigned a corresponding suffix. A different symbol for 
the finite-difference operator is used to distinguish these three cases: 

fi-fo = *fo) 

= V/i . (4.5) 

— VfJ 

This is generalized in the following three schemes for a difference table: 


x / 
X-lf-l 

X-\f -1 


4 /-, 

4 /-, 


47 -. 

47 -. 


47 -, 

47 -, 




/ 

/-a 

fo 

/. 

/a 


«/-* 

«/.j 


47 -, 

47-1 

47 . 

« 7 , 

47 , 


47-.J 

»r-\ 

47 J 
47,1 


«7-* 

47 -. 

47 . 

47 , 

47 . 


Central ditforenres 


In any particular numerical case the numbers will be the same in each 
table; what is different is the general notation for these numbers, the 
notation which expresses the value of x with which each difference is 
associated. 

Differences with the same suffix value in table (u) are called ‘forward 
differences’; they lie on a downward-slanting line on the table, such as 
those underlined. The forward differences from the first entry in a table 
are sometimes called leading differences’. Those differences with the 
same suffix value in table (b) are called ‘backward differences'; an 
example is indicated similarly. Those with the same suffix in table (c) are 
called ‘central differences’. 

Central differences are much the most useful in practice. Many 
formulae in central differences involve onlv alternate orders of differ- 
cnees, whereas the corresponding formulae in forward or backward 
differences involve all orders of differences; also the coefficients of 
higher terms in cent ral-difference formulae usually decrease more rapidly 
with the order u of the differences than do the coefficients in formulae 
involving forward or backward differences. Further, this notation gives 
a much more natural relation between finite differences and derivatives. 

t It »> sometimes convenient to distinguish l>utworn (ho general symbol hx for tho 
int'.TMil length and tho particular value which it has in a particular calculation. 
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In the analytical work of deriving formulae for interpolation, integra¬ 
tion, etc., in terms of differences, use of forward differences leads to 
rather simpler algebra; but in order to get from the results the central- 
difference formulae which are most convenient for practical use, it may 
be necessary to do some rather laborious algebra, which may then onl\ 
give the coefficients of the central-difference formulae term by term, and 
be difficult to generalize to give the general term. It seems best to work 
throughout in terms of central differences, and so obtain directly the 
formulae for interpolation, integration, etc., in the forms in which they 

are most useful for practical work.f 

It will be seen that in the central-difference scheme (c) on p. 38, only the 
even-order differences have integral suffixes. It is sometimes convenient 
to take the arithmetic mean of two adjacent differences and to write 

M8/o = W*+S/_ t ) 
and in general ^ = $(S«/, +1 +$»/,_,) 

= KS-Vi+i-S"-7;-i)- 

Then the available differences arc odd-order differences with (integer-f ‘>) 
suffixes, and even-order differences and odd -order mean differences with 
integral suffixes. A set of successive function values fj from j = J— k 
to J+k inclusive is said to be ‘centred on’ the argument value x d or on 
the function value fj ; similarly for a set of differences 8 n / ; . 

4.3. Finite differences in terms of function values 

It is sometimes convenient to have differences expressed in terms of 
the function values from which they are derived. We have in succession 

8 /, = < 40) 

8' 2 /o — 8/, —8/_, — (/,—/o) — (/o — /-i) 

= /i-2/o+/-i • < 47 > 

sv, = sv.-sv. = (A- 2 /.+/o)-(/i-2/o+/-.) 

and in general 8 *f t = ^ < — k'.ln —* 4 ' 8 * 

A - 0 

as cjyi be proved by induction; for an alternative proof sec §4.G. The 
coefficients of the function values in 8"/; are those in the binomial 
expansion of (1— z) n . 

t Tito symbol A is then loft froo for anothor uso. to indicate the difference between the 
data or between the resnlts of two similar calculations. 
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In particular, the differences of the function 

f m = 0 rn # 0 

/o = 1 

are the binomial coefficients: 


/ 8 / 87 87 87 

/\ n 1 



The effect of an error € in a function value on the difference table builds 
up in the same way: 

i- 4 < 

0 +6e 

i-«« 

*+* 

This is the basis of an important application of differences to checking 
tables, and sometimes for correcting isolated errors, which will be 
considered shortly (§4.42). 

4.4. Simple applications of differences 

The simplest applications of differences are: 

(а) Building up polynomials; 

(б) Checking tables; 

(c) Smoothing. 

Of these, (a) and (6) will be considered here and (c) in § 11.4. 

4.41. Differences of a polynomial 

f 

An important property of finite differences is that for a polynomial of 
degree n, the «th order differences are constant. One proof of this is by 
induction. 

Suppose that 



8 m (.r m ) = m! (8.r) m 


(4.9) 
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for all integral values of m up to m = n, say; then it will be proved that 

(4.9) holds for m = ft+2. Since for p <m 

(4.9) implies that 8 m (xP) = 0 for p < m. (4.10) 

Now from (4.7) 

§2(3.71+2) = (a;-}- 8 x) n+ 2 —2x n+2 -f {x- 8 x ) n+2 

= ( n + 2 )(n+l)( 8 x) 2 a; n +terms of lower degree, 
so 

8 n+2 x n+2 = 8 n (8 2 x n+2 ) 

= (n+ 2 )(n+l)( 8 x) 2 8 "[x n -fterms of lower degree] 

== («+2)(n+l)(8z) 2 8"(x") [by (4.10)] 

= (»+2)(»+l)(8x)*n! ( 8 x) n [since (4.9) holds for m = n] 

= (n+ 2 )!( 8 x)"+ 2 . 

Now (4.9) holds for m = 1 and m = 2, hence the induction succeeds, and 

( 4 . 9 ) holds for all integral m. 

It follows that for a polynomial of degTee m, say 

pjx) = a Q x m +a l x m ~ x +... i 

the mth differences are constant and equal to a Q m \ ( 8 x) m . 

Another derivation, which depends on some properties of a set of functions to 
which reference will be made later, is as follows. Consider the polynomials^ 

M) = 1 

0 m (£) ="n tf+W”*- 1 )-*)]; < 4 - n > 

k-0 

0 M (|) is a polynomial of degree m in f, and its argument value is the mean of the 
extreme factors. The first few such functions are 

0 o(£) = b 

iM*) = 

*<*)-tf+!)«*- ! >-ft**- 1} . 

For intervals 8£ = 1, the first differences of ip m are 

= [^d- 4(m-hl)][^4-4(m— i(m —3)] — 

-[ g + K ™- 1 )]•♦•[£- 1 w * 


f Sometimes callod ‘factorial polynomials’. 
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The common factor of the two terms, indicated by a bracket, is a polynomial of 
the set (4.11); it has (m- 1) factors and the mean of its extreme factors is (£ + *), 
so it is 0 m _,(£ + £)• Hence, for m > 0, 

80 m (£*H) =* [{^+i(” l + 1 )}-^-i( m_1 )^m-i(l+i) 

= Hn-llt+i)’ (4 ' 12) 

and 80 o ^ 0. 

Repeating the operation wo havo 

8 2 0m(£) = 80,„(£+i) —80 m (£—J) 

= ™[0m-i(£ + i)“ 0m-l(£ — $)] 

= 

and ultimately S m 0 m (£) = m! 

S m+, 0m(£) = 0. 

Any polynomial pjx) = a 0 x m + u l x m - 1 + ... tabulated at intervals (8*) in x, can 
bt written p m (x) = er o (8x) m [0 fI ,(£) + 6i0 m _i(£) + & 20 m- 2 (£)d'*“] 
where £ = x/8x, so 8> m (x) = a 0 wi! (8x) m 

as already shown. 

This result. that the with differences of any polynomial of degree wi are constant, 
and its (m+ 1 )th differences are zero, corresponds, in finite differences, to the result 
in differential calculus that the with derivative of such a polynomial is constant and 
its (m -{-1 )t h derivat ive zero. The functions 0 m (x) take the place, in finite differences, 
of tlie function x" in differential calculus, as the polynomials whoso form remains 
unchanged on differencing. 

These functions will appear later in another context, for which some further 
properties of them will bo required. From the definition (4.11) it follows that 

lM-*> = (-rV'mtf). 

Hence for odd values of m 

02n ,l(£) + 02U4.|( 1 -£) = 02.14 l(£)~ •/'In h(£- a 

= (2n*f 1)02„(£—4) (4.13) 

by (4.1 2), whereas 

02 n+i(£) 02 n+i(l £) = 02n+l(^)4*02n+l(^~ 

= [(£+»0 + (f-n-l )]0 2 „(f-J) 

= (2£-l)0 s „(f-l)- (4.14) 

4.42. Building up polynomials 

The constancy of the with differences of an with order polynomial can 
ho used to construct a table of values of the polynomial by building up 
successively the lower orders of differences from tlie higher by repeated 
addition. It is necessary to calculate at least m function values to give 
a set ol leading differences from which to start the construction of the 
difference table, and it is advisable to take one or two more to provide 
a check. 
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Example : 

To evaluate the polynomial y = 

x 3 — 5x 2 

-f 6x-r 1 for x = 

0(1)10: 

X 

x 3 — 5x l + 6* 

+ 1 = 

y 




-2 

1 

CD 

1 

to 

o 

1 

IO 

+ 1 = 

— 39 """ 

"''+28^' 



-1 

<x> 

1 

o 

1 

1 

+ 1 = 

— 11 

12 

— 16^~ 

6 

0 

ooo 

+ 1 = 

1 


— 10 

start 





2 


6 

I 

1 -5 6 

+ 1 = 

3 



6 

2 

8 -20 +12 

+ 1 = 

1 

0 

2 

6 

3 



1 

8 

8 

6 

4 



9 

22 

14 

6 

5 



31 

42 

20 

6 

6 



73 

68 

26 

6 

7 



141 

100 

32 

6 

S 



24 1 

138 

38 

6 

9 



379 

182 

44 


10 

1000 -500 +60 

+ 1 = 

: 561 



check 


Here five function values, from* = — 2 to 2 (the simplest ones to evaluate) huvo 
been calculated to provide a start for building up the differences. We know from 
(4.9) that the third differences must have the constant value 6, and this provides 
u check on the starting values. From the constant third differences of G the 
second differences an; built up, then the first differences, and finally the function 
values. The function value at x = 10 is easy to calculate directly, and is so calcu¬ 
lated to provide a check on the successive additions. 


It will be noted that intermediate values of y are calculated by addition 
only: this process can be carried out very effectively on an adding 
machine fitted with a printing mechanism (§2.20). For example, in 
summing the second differences to give the first differences, after adding 
each second difference the resulting value of the first difference, which 
is the current total, is printed without clearing by taking a 'sub total'. 
The results appear in the form of alternate values of second differences 
and first differences; the former can be checked against the table of 
values and the latter then summed similarly to give the function values. 

It is necessary in using this process to keep all figures without rounding 
off, although final results may not be wanted to this accuracy. 
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Example: To evaluate the polynomial y = x* 
decimals required. 


-5x*+6x+l iorx= 0(0*0I)0*l;four 


X 

•002 

0-01 

0 

001 
002 
0-03 
004 
005 
006 
007 
003 
009 
0 10 


—0000,08 

— 0000,01 

0 

+ • 0000,01 
+ 0000,08 


-5*’ +6 x +1 

- 0020 -0 12 +1 

-•0005 - 0 06 +1 

0 0+1 

- 0005 +0 06 +1 

- 0020 +012 +1 


“ V 
= - 8779.92 

=> 0391.99 

= 1 0000,00 

= 1 0593,01 

- 11180.08 

1 - 1755,27 

1 2320,61 

1 - 2876,25 

1 - 3422,16 

1 - 3958,43 

1 4185.12 

1 5002,29 

1 5510.00 


61507 

60501 

59501 

58507 

57519 

56537 

55561 

51591 

53627 

52669 

51717 

50771 


I'V 

-1006 

-1000 

- 991 
--988 

- 982 

- 976 

- 970 

- 984 

- 938 

- 952 

- 916 


y rounded off to 
four decimole 

•8780 


0 

C 

0 

6 

6 

6 

6 

6 

6 

6 


•9395 
1 0000 
1 0595 
11180 
1-1755 


1 4485 
1-5002 
1 5510 


615 


605 

595 

535 

575 


527 

517 

508 


- 10 
- 10 
- 10 
- 10 
- 9 



- 10 
- 9 


Note*: (i) Although the third difference of 6 in the sixth decimal is smaller than 
the rounding error in the four-decimal values finally required, it must not be 
neglected on that account, as this would be a systematic rounding error which would 
accumulate and ultimately affect the results wanted. Omission of it would be 
equivalent to omitting the x 3 term in the polynomial, and the error would already 
be 10 in the fourth decimal at a: = 0-1. 

(ii) Here a typical copying mistake (78 for 87) has been made in the column of 
rounded-off values, which are those finally required. It is a mistake which is easy 
to make at this stage; all the calculations have been done, and all that is wanted 
is to copy the four decimals required with the appropriate rounding off; uncon¬ 
sciously one may relax somo of the care with which the rest of the calculation has 
been carried out, and then a mistake of this kind can easily occur. Such a mistake 
is easily identified by differencing the rounded-off results and such a check should 
always be used. As will be seen in the following section, the irregular differences 
not only locate the erroneous value unambiguously, but strongly suggest the 
correction. 

4.43. Checking by differences 

We have seen in § 4.4 that an isolated error e in a function value makes: 
a maximum error <= in the first differences 


2<r 

second differences 

3e 

third differences 

G<? 

fourth differences 

10c 

fifth differences 

20e 

sixth differences 
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whereas the magnitude of the differences themselves normally decreases 
with the order of differences; if it does not, the function is not well defined 
by the table. Hence an error shows up more and more as the order of the 
differences is increased. Examination of the differences of a function is 
one of the best checks against random errors; it will not necessarily check 
against systematic errors. 

The differences which are affected by an error spread fanwise from the 
incorrect function value (see § 4.3), and this can be used to locate an error. 

Example: 

x V 

0 358 

1 370 

2 397 

3 451 

4 643 

5 683 

6 880 

7 1124 

8 1476 

9 1888 

10 2383 

11 2965 

12 3637 
The last column is 18x(l, —4, 6, —4, 1). 

Notes: (i) The existence and location of an error is unambiguously shown by the 
table. 

(ii) A change Ay in a function value y makes changes (1, — 4, 0, — 4, 1) times Ay 
in successive values of the fourth difference, centred on the changed value of y. 
A few trials show that a change Ay = +18 will make all the fourth differences - 1. 

The error can often bo corrected in this way. 

(iii) A transposition of two adjacent digits differing by m will produce an error 
of 9m in terms of the less significant of the digits as unit. It has already been 
mentioned that transpositions form a common type of mistake; values of Ay which 
are multiples of 9, or nearly, probably arise from mistakes of this kind. This can 
be checked from the values of the digits involved. Here Ay = 18, hence m = 2 
in the last figure. The value y = 1124 at x «= 7 should read y = 1142. 
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(iv) In this case the fourth difference of the corrected table is exact; the location 
and correction of the mistake is not affected by rounding errors. 

Example: Here the values of y arc alleged to be rounded off from a table of x*: 






Correction Revised 

Correction 

Corrected 

X 

y 

Si/ 

S ! y 

to 8 2 y 

8 2 y 

to 8 2 y 


38 

61644 

806 






39 

•2450 

796 

-10 


-10 


-10 

40 

•3246 

785 

- n 


-11 


-ii 

41 

•4031 

776 

^-0 


-9 


-9 

42 

•4807 

"740^ 

-36 

H27 l 

-9 


-9 

43 

•5547 

__785^ 

+ 45 

-54 

-9 


-9 

44 

•6332 

750 


-f 27. 

-8 


-8 

4f» 

•7082 

^750 


-9x 

-9 


-9 

46 

•7832 

745 

— •> 

+ 18 

+ 13 

— 20\ 

-7 

47 

•8577 

^705^ 

-40 

- 9; 

-49 

+ 40 l 

-9 

4S 

6-92s 2 

718 

^4d3 


-} 13 

— 20/ 

-7 

49 

7 0000 

711 

— 1 


— 7 


-7 

50 

0711 

703 

- 8 


- s 


-8 

51 

■1414 

697 

-6 


— G 


-G 

-1 

o — 

■211 1 








.Yoft.v: (i) Herr a succession of sc-ven values of S\y is irregular. The first obviously 
wmug value. —36 at .r = 42, indicates a mistake at x — 43; the value of 8 : y(42) 
would be expected to he — 8, — 9, or — 10: that is. the correction is -f 28, -f-27, 
or 26. I lie value -f 27 suggests a transposition of two digits differing by 3, and 
r> ference to the function values shows that tlie end digits do differ by 3. 

fii) ( orrection of this mistake then makes the differences smooth, apart from 
.-light ii reguluriti. which can be ascribed to rounding errors in the function values, 
as fjir as o-’.i/l 1*) inclusive. The nextfour second differences are irregular, indicating 
Mistakes in both the values i/(45) and i/(4G). The value of 8^(45) would be expected 
o be s or 0; that is the correction is -8 or -9; the latter suggests an inter¬ 
ning, oft w., digits, littering by l.and when the corresponding correction has been 
1111 ,1 1' ' ,v '' have t lie series of second difterencesgiven in the column headed 1 Revised’. 

bn i 1 he ic xt t hr.-, second differences should be about - 8, and to give them all 
ihb vain, we would require corrections (- 21. -MI, -21); the corrections arising 
n .m a -ingle change in y must be in the ratio (I : — 2; 1). so the error in y(47) is 
1 firmer would he produced by doubling the wrong one of two 

digiis d.If, ring by 2. and as such digits do occur in y(47) in the right, place, the 
t rn»r can i»o a sunbed to this cause with fair Certuintv. 
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These examples show that it is possible to use differences not only for 
detection and location of errors in tables, but for correcting them, when 
the nature of the error is clear from the behaviour of the differences, or 
for indicating a probable correction when it is not. In the case of the 
second example just given, it would of course be much better to use the 
differences simply to indicate the erroneous values, and to refer back to a 
table of x * to correct them. 


4.44. Effect of rounding errors on differences 

In most tables almost every function value will be in error to some 
degree, on account of rounding errors. Although the rounding error in a 
function value may not be more than \ in the least significant figure, the 
effect of an error is exaggerated in the higher differences, which un¬ 
avoidably become somewhat irregular, and the more so the higher the 
order of differences. It is important to realize this, otherwise irregularit ies 
in differences which arc due to rounding errors may be taken as indicating 
mistakes, and time may be spent trying to find mistakes and to make 
changes in function values which cannot be improved except by taking 
more significant figures. 

The greatest effects of rounding errors will occur when alternate func¬ 
tion values are rounded off by + i and — ?> alternately. Then depai tut es 
of the nth differences from those for unrounded function values may be 
U p to ±2 n ~ l in the last place tabulated and alternate departures will be 
of alternate signs; though such large irregularities will be rarer the higher 
the order n of the differences. It is useful to have a working criterion for 
the magnitude of the fluctuations in the different orders of differences 
which can be expected as the result of rounding errors. Connie t gives 
the following limits for various values of n: 


n 


1 

±1 


*> 

±2 


3 

±3 


4 

‘ G 


5 


±12 


6 

i 22 


S 

4-HO 


10 

i 300 


Differences having fluctuations less than those limits can be accepted; 
only those having greater fluctuations should be regarded as suspicious. 

The example overleaf illustrates the way in which irregular differences 
may occur in the most accurate rounded-off values of a smooth fund ion. 

From examination of the differences in the table, one would be very 
inclined to ‘correct’ the rounded values of y{o) and y(S) to 3!) 3 and < < < 
respectively, giving the third differences shown on the extreme right ; 
but reference to the exact values of y shows that this would be incorrect . 

t Chamber* a 6-1'"jure Mathematical Table*, vol. 2 (1949), Introduction, j>. x.xxi. 
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Values of y rounded 

* Correc¬ 








off to nearest unit 

tions' 


X 

y 

8y 

8 *y 

S’y 

8 *y 

y 

8y 

S*y 8 s y 

to 8*y 

8»y 

0 

61-24 

4581 




61 

46 





1 

107-05 

5478 

897 

109 


107 

55 

9 

1 


1 

2 

161-83 

6484 

1006 

162 

53 

162 

65 

10 

1 


1 

3 

226-67 

7652 

1168 

214 

52 

227 

76 

'11 

4 

-1 

3 

4 

303-19 

9034 

1382 

265 

51 

303 

91 

15 

0 

+ 3 

3 

6 

393-53 

10681 

1647 

315 

50 

394 

106 

15 

6 

-3 

3 

6 

600-34 

12643 

1962 

564 

49 

600 

127 

21 

1 

+ 1 +1 

3 

7 

626-77 

14969 

2326 

412 

48 

627 

149 

22 

7 

-3 

4 

8 

776-46 


2738 


47 

776 


29 


• 




17707 


459 



178 


2 

+ 3 

5 

9 

963-53 

20904 

3197 

605 

46 

954 

209 

31 

6 

-1 

5 

10 

1162-57 

24606 

3702 

650 

45 

1163 

246 

37 

5 


5 

11 

1408-63 

28858 

4252 



1409 

288 

42 




12 

1697-21 





1697 







This example illustrates that smoothness of differences of rounded 
values of a function is not a guarantee that these values give the best 
representation of that function. The adjustment of function values by 
differences cannot be depended on to ±1 unit in the last place; it is 
possible, as in this example, to make the differences over-smooth.f 


4.45. Direct evaluation of second differences 
It is sometimes convenient to be able to evaluate second differences 
directly from function values without the intermediate step of calculating 
hrst differences. This can be done on a machine as follows. 

Suppose first that the second differences of f are positive; S 2 ^ is cal¬ 
culated from the formula 


S2 A = /j-i+/j + i-2/,, 

the terms being taken in this order; then/, is set ready for the calculation 

^‘fj+ i = /* + fj+z—2fj+i> 

and so on. 

If the secor, 1 d’fferences are negative, this process will give them in 
complementary form; then it is more convenient to obtain 


-SV> = 2 

t For a further discussion of chocking by differences), Bee J. C. P. Miller, M.T.A.C. 
4 (1950), 3. 
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the terms being taken in this order so that when —8 2 / ; has been obtained, 
f }+1 is already set for the calculation of 

& 2 fj +1 = ~fj+i fi fj+2- 

If the function values are negative, their moduli are set, and the signs 
of the machine operations altered accordingly. 

This is a useful process for checking values of a function built up from 
second differences by summing the second differences to form the first 
differences, and then summing first differences to give the function 
values. The direct calculation of second differences provides a good check 
of these two successive summations. 


4.46. Building up from second differences 

A function can be built up directly from its second differences, without 
calculation of the first differences, by a process which is the converse of 
that of the previous section. If the function is positive, we have 

fj+i = SVi*. 

this is transferred to the setting levers, and used in the first step in forming 


fj+2 = 2 fj+l— /j + S2 /i+l* 

If fj is negative, it is more convenient to form 

(-fi* i) = 2 (-/>)-(-/>-x)- S2 /r 

If this process of building up a function from its second differences is 
used, the method of the previous section should not be used for checking, 
the processes are too nearly alike for one to be a good check o t le lesu 

of the other. , . , 

One machine, the Brunsviga 20, has two facilities which are very 

convenient for building up a function from its second differences, these 
are transfer from the accumulator to the setting levers and an arrange 
ment for clearing only the right-hand half of the accuniu a or, earing 
left-hand half unaffected. The latter feature is known as split clearan e , 

and has the effect of furnishing the machine with two registers 

In the present application, the first differences are accumulated in he 
right-hand half of the accumulator (R.H. for short) an le 
itself in the left-hand half (L.H.) Let S/,_, be in R. H and f in L H hen 
8 V/ is set on the setting levers (S.L.), and added into R.H., * 
contains 8 f IH . This is transferred to S.L., and the operation of,clearing i 
from R.H does not affect L.H. ; it is added back into R.H. and also, after 
shifting the accumulator, into L.H.; R.H. now contains 8/, , andI L.SL 
contains / <+1 . The accumulator is now shifted back, 8»/ J+ , set, and the 
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process repeated. The only quantities needing setting are the second 
differences 8 z fj. 


4.5. Differences and derivatives 

We have seen that functions defined by analytical formulae are ade¬ 
quately represented by tables only in ranges away from singularities and 
discontinuities, and that if a table is the only information we have about 
a function, we may regard the function represented by the table as being 
differentiable as many times as we require. We will therefore suppose 
that in any applicat ion of numerical methods to functions specified by a 
table, the function can be expanded in a Taylor series over the range with 
which we are concerned. 

Then we have 

f ± i = /o± (M/i + £ (S*)To ± ^ (8x) Yi"+... (4.15) 

and in general 

f±n = fo±(n S-r )f' 0 + j } (n Sx)r o ±^(n S*) 3 /"+..., (4.16) 


* 2 /o=/i-2/„+/- x = 2 


+ 0(hx)* 


(4.17) 


the remainder term being of order (8x) m if the series is cut off after m 
terms. We shall only derive a few relations directly from these expansions, 
as we shall shortly see a quicker and more effective way of deriving rela¬ 
tions of the kind we require in practical work. 

Substitution of series such as (4.15), (4.16) into the formulae giving 
differences in terms of function values gives a set of relations for 
diffi voices in terms of derivatives] for example, if terms of order (8x) 6 are 
included, 

= i^r-uz ++o(8.r)*, 

and similarly 

»Vo - / 2 -4/ 1 + 6/ 0 -4/. 1 +/. 2 = (&c) 4 [/4 r +J(&c)s/?] + 0(8ar)«. 

(4.18) 

Fiom the symmetry of the coefficients in formula (4.17) it follows that 
even-order differences 5-"/o involve only even-order derivatives at a* 0 .. 
It follows from (4.17), (4.18) that 

Um[b% (&r) a ] = /o, 
lim [SVo,'(&t) 4 ] = f' 0 '\ 

OJ-M) 

and similarly for higher orders of differences; thus finite difference 
ratios are closely allied to derivatives. But in using differences, it is the 
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differences themselves that enter into most formulae, rather than 
difference ratios. 

The relations (4.17), (4.18) and similar ones for higher-order differences 
can be regarded as equations for derivatives in terms of differences, and 
solved for these. A more important relation, however, is one between 
the second difference of/ and its second derivative and its differences. 
From (4.17) applied to the function/" we have 

8 2 /" = (Wf+iyM*/?] +0(Sx)*, 

‘ 8 % = (8*) 4 /S f +0(8x)«, 

80 thafc (8x)% v = 8To-&To+0(8x)«, 

{*x)*fg = sy o+0(8x)*, 

and substitution into (4.17) gives 

8"-/o = + + < 4 - 19 ) 

Similarly, 

m */. = «/,-/_) = («*)[r.+ 3 ‘, (s*w:+5, <«*)*/!]+o(s.)' (4.2«) 

and application of (4.17), (4.18) to /' gives 

5 2 /o = (&€)•[/?■+ 1 , 4 (8*)*/51 + 0(8*) 6 , 

8 4 /o = (8*) 4 /5+0(8ar)«; 

and substitution in (4.20) gives 

mS/o = (^)[/o+ J eS 2 /;-,Jo5y;H-0(8.rr. (4.21) 

As we shall see later, the first two terms in the square bracket here give the 
formula usually known as ‘Simpson’s rule’ for numerical quadrature. 

For relations involving even-order differences, and odd-order mean 
differences, the expansions (4.15), (4.10) in/and its derivatives at x = -» 0 
is the most convenient. For corresponding relations involving odd-ordei 
differences and even-order mean differences, it is often more convenient 
to expand in terms of / and its derivatives at x = x j. 


4.6. Finite difference operators 

A powerful method of obtaining formulae for interpolation, integra¬ 
tion, etc., in terms of finite differences is by means of finite difference 
operators. We have already recognized that the symbol 5 or A prefixed 
to a symbol representing a function can be regarded as representing an 
operation performed on that function. \\ e will now extend this idea, and 
first define some further operators. 
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The operator E is defined by 

Ef(x) = f(x+Sx) 

or shortly Ef i = f j+1 . (4.22) 

This operator advances the argument from one value to the next of the 
finite difference table, and is sometimes called the ‘shift operator’ or 
‘forward shift operator’. Its inverse, written E~ x or 1/E, the ‘backward 
shift operator’, steps the argument back from one value to the previous 
one in the difference table; that is 

B-'fj = fi-1 - (4.23) 

If D is the differential operator D = d/dx, Taylor’s expansion can be 
written symbolically 

fi = /(* o+ 8;c ) = e (8l)D / 0 , 

so that, formally Ef 0 = e (8x)D / 0 (4.24) 

for all functions / for which the right-hand side is significant. A relation 
such as this, between results of different operations, which is independent 
of the function / operated on, is often written as a relation between 
the operators, without an operand explicitly indicated. We follow this 
usage, and, in accordance with it write (4.24) as 

E = e (8x)Z) . ' (4.25) 

Two operators of which we shall make considerable use are E * and its 
inverse E ~*. E 1 is the operator which, applied twice to gives /,, 
independently of the particular form of the function /; that is to say, it 
is an operator such that for any operand/, 

= /. = E So- 

It i; !ear that an operator which advances the argument value by half 
the tabular interval satisfies this condition; that is 

&/{x) =/(*+*&*) 

or £‘/o = /,. (4.26) 

From Taylor’s series = <? J(8 - r)D / 0 

which is consistent with (4.25). 

The ‘forward difference operator’ A is defined by 

A/(*) =/(*+&*)-/(*) = Ef(x)-}(x) 
or shortly A/„ = /,-/„ = (E- 1 )/„ 

which, expressed as a relation between operators, is 

A = E-1 . 


(4.27) 
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The ‘backward difference operator’ V is defined correspondingly by 

V/(ar) = /(*)-/(*-«*) 

or V = 1 — E~ x — (E—l){E. (4.28) 

The ‘central difference operator’ 8 is defined by 

&f(x) = f{x+\hx)—f{x—\hx) = (E*—E-i)f{x) 

which, expressed as a relation between operators, is 

8 = E*-E~K ( 4 - 29 ^ 

Another useful operator is the 'averaging operator’ p, defined by 
p/(x) = i[f(x+i&x)+f(x-iSx)] = )>[EWx) + E-mx)], 

i. e . f = «*+*-*>. (4 ' 30) 

These operators all have their inverses. We have already considered 
the operator inverse to E. The operator inverse to 8 is the central sum 

operator’ a = 8 -1 defined by 

af n = cr/n-i+Zn-I 
or o=E>l(E-l). 

It should be noted that of, like an indefinite integral, is undetermined to 
the extent of an arbitrary additive constant. The operator inverse 

ix will be considered in § 5.2. . f . 

These operators are all linear; that is to say if 0 is any one of them, 

and / and F are any two functions, then 

0(f+F) = Of+OF. 

The operators E, A, D, 8 , and V are also cojunjutaip; that is, if 0, 
and 0 2 are two of these operations and / is any funct 10 , 

0 ,( 0 ,/) = 0 ,( 0 ,/). 

a and 8 are not necessarily commutative, since »(«/) may 

8 ( 0 /) by a constant, just as J (d//dx)dx may differ from / by ,co-starX. 

Some useful relations miy be obtained from (4.23) to (4.30). 

example, from (4.29), . 32 __ E — 2-\-E~ l 

ir f C 2 f _ f _of xf ) and from (4.30) 

(the operational form of 5 2 /o — Ji ~J°' 

^ = \(E + 2 + E~ l ), 

whence = ^ . 

p*=i+^- <4J1) 


whence 
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And if in (4.29), (4.30) we substitute for E from (4.25) we obtain the 
formal relations 


(4.32) 

(4.33) 


(4.34) 


8 = 2 sinh l(8x)D, 

/z = cosh |(8z)Z). 

Also we have 

(#4-1)8 = #*(#*-}-#-*)(#*-#-*) = (#*4-#-*)(#-i) 

= 2(E-l)fM. 

Also 8” = [#-*(#-1)]« = E~* n (E- 1)", 

so that &»f. = (E-irE-l'f, = (E-l)%_’ n -, 

expansion of (E —l) n by the binomial theorem gives 

57; = y (-1)* _ E'-Kfi . = V (-1)* ” ! f 

ifk k'{n-k)l Jl in 2 / } k\(n—k)'/ j+ln ~ k 

A=> 0 

in agreement with (4.8). 

■\\ e shall make considerable use of relations between operators, such as (4.32) and 
(4.33), which imply the use of Taylor’s series in the form 

e<7(*) =/(*+£) (4.35) 

without a remainder term. However, in using the formulae we finally obtain by 

means of these relations, wo shall in almost all coses retain only the first few terms 

therefore making truncation errors in which the remainder term of the Taylor 

expansion can be considered as incorporated. In most cases an analysis of the 

truncation error and its relation to the remainder term in the Taylor expansion 

can be earned out by the method of § 0.7. But it will be as well to know for what 

K,m,s "motions this expansion can be used in tho form ( 4 . 35 ). 

1. I'ohjnommh: it, « clearly exact for polynomials since the series terminates, 
“• L.iponcntiah: if/(j) = c ' ,r , then 

I'rTfhmv l,nKk " C0nwr »“ for 0,1 «■«*• off and «, and its value 

= c n tc nx = =* f( v +£) 

" i,hout restrictio - - »which t„e 

3- Product* of exponential* and polynomials: wo will prove that if 

r<"c(.r) = r(.r-f£) 

' l , ‘ .-(.»•)] =-. (.r-t £);(.*• 4-£) 

^Id^ 11 then it can be applied to 

Ph' '< any polynomial. We live ^ ** aPI * Mcd to 


f' * Jr: 






: «'.* 2[rD2= b2Dc]-f ~£ 3 [xD 3 z f 3 D'z] j-... 
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Since z is assumed to be such that (4.35) is valid, the first square bracket is 
xz(x + g), and the second is £z(x + £). Hence altogether 

e(°[xz(x)] = (x + ^)z(x + i). 

Thus (4.35) can be applied to products of exponentials (including circular 
functions) and polynomials; and, since the operator is linear, it can be extended 
to sums of products of exponentials and polynomials. 

4.7. Examples of the use of finite difference operators 

It is convenient, for brevity, to have a single symbol for the operator 
(8 x)D\ this will be written U, that is 

U = (8.r)D. ( 4 - 3G ) 


Then the relations (4.25), (4.32), and (4.33) are 

E = e u , 8 = 2sinh hU, /* = cosh \U y (4.37) 

so that u = 2 si nil -1 ’8 < 4 - 38 ) 

= [(sinh- 1 £8)/AS] 8. ( 4 - 39 ) 

Since 8/ = {Sx)Df+ 0(Bx) z for any particular / to which these relations 
between finite difference operators can be applied, it follows that in 
expanding these and other relations in powers of 8 or 1,8 or L can e 
regarded as a quantity of order (bx) n . 

4.71. Derivatives in terms of differences 

Taking the nth power of both sides of (4.39) we have 

U n = [(sinh- 1 A8)/.\S]"8". ( 4 - 4 °) 

Since (sinh- 1 *)/* is an even function of c, this expresses= iSx)”D* 
in even powers of 8 if n is even, and in odd powers o i « is ot < 1 

available central differences 8"/> of cren-orcler have integral values o j. 
whereas those of odd-order have (integer4-.1) \ allies o j. c,lce ' 
relation can be used to obtain expressions lor m/i-oukr cm a \\a> a 
tabular values, D 2m f jt or odd -order derivatives half-way bttuxin tabular 

values. 

An alternative form is 

U" = [/x _, {(sinh _1 '8 )/i8}"l/i8». ( 4 - 41 ) 

Since the relation between ^ and 8 is — 1-1 i 3 > t | K o\ki.i\ 11 
square bracket is still an even function of 8, so that lor «frf values o 
this expresses U"f in terms of odd -order mean differences h f, " 1 <- 
are available at tabular values; hence this is the useful formula loi od, - 

order derivatives at tabular values. 
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The expansions of (4.40) and (4.41) both for positive and for negative 
values of n can be carried out by taking the series for (sinh -1 £8)/|8: 
(sinh -1 18)/48 = 

and raising it to the appropriate power; and in the case of (4.41) multi¬ 
plying also by the expansion of /x" 1 = (l-f|$ 2 )-*. General expansions 
for (U/8) n and [((7/8) n //z] as far as 8 10 for any value of n have been 
given by Bickley;f taken to terms in 8 8 that for (£7/8) n is 

l u \ n _ /sinh"48\ n 

l / 

= 1 71 S 2 , 5ti 2 +22 w S 4 357i 3 +462n 2 -f 1528n„ , 

24 5760 2903040 b + 


l_75 ^+4620n3- f 40 _ 724n 2 +119856n 

13934,59200 -\-upx) . (4.4Z) 

For positive n, the three cases of this formula which we shall need here 
are the cases n — 2, 4, and 6 , namely, 

( U/S ) 2 = [l-A8*+A8 4 -5l 5 8 6 ]+0(8*)«, (4.43) 

(U/8) 4 = [1-IS 2 +«Io8 4 ]+O(8z)0, (4.44) 

(W = [l-iS 2 ]+0(S^. (4.45) 

For odd positive powers of ((7/8), the only important case is n = 1, for 
which. 

(Uif.8) = [I-J 8 2 + ;i ’ 0 8 4 - 1 ? 5 8 6 4- 9 J 5 8 8 ]-f 0(8*)*®. (4.46) 

These formulae, which give powers of U = ( 8x)D in terms of 8 , are 
operational forms of formulae for differentiation, since, applied to a 
function /, they give D n f in terms of the differences of/. They have, 

however, other and more important applications as will be seen in the 
next section and subsequent chapters. 


4.72. Negative powers of (i// 8 ) 

Other important relations are some involving negative powers of 
( 5). Jne way of obtaining these is by use of formula (4.42) orBicklev’s 

corresponding formula for (W/* with negative values of n, for which 

ic se formulae are also valid. For example, substitution of n = -2 in 
(4.42) gives 

"[’-K'iS’-ildS'UW'-iM+OI&r)”, (4.47) 

.■ nofher procedure is first to express the operator in terms of U, expand 
as a series m l . ard -hen substitute in terms of S from formulae ( 4 . 43 ) to 
( ■ 6 ), this only involves the use of formula (4.42) for positive values of a. 

hiv 'tloy, Journ. Math, and Phys. 27 (1948), 183. 
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The main operators which involve inverse powers of U and for which 
we require expressions in terms of 8 are ( 8 /C/) 2 , ( 8 //xC/), and (fi8/U). In 
terms of U they are 

( 8 /C /) 2 = [(sinh |C/)/iC /] 2 = 2(cosh U-l)/U 2 , (4.48) 

( 8 //xC/) = (tanh iC/)/i!7, (4.49) 

(/x8/C7) = (sinh U)/U. (4.50) 

Expansion of (4.48) in powers of U gives 

(S/I7)= = l+^ 2 + U‘+0$x)> 

= l+*S 2 (C7/S) 2 + s JoS 4 (^/S) 4 -H. 2S l si> S 6 (t7/S)«4-«9(Sjr)6. 
Substitution from formulae (4.43) to (4.45) then gives, to terms in 8 6 , 
(8/C/) 2 = 1 + ig8 2 [ 1 — i\8 2 +aoS 4 ] + oeo8 4 [ 1 — £8 2 ]++ 0(§x)* 

= [1+A» 2 -*1 fl8 4 +«RW # ] + °( 8 *) e 

in agreement with (4.47). 

Similarly the following expansions can be obtained: 

( 8 //xC/) = [ 1 — I’gS 2 -f- 7 V 6 S 4 —H- ai rAVoo5 8 ] 4- O(Sor) *°, 

(/x8/C/) = [l + i8 2 - i ioS 4 + i5 , i2 86 -5-JbooS 8 H- 0(hx)'°. 

The latter of these can alternatively be obtained from the former by 
multiplying by /z 2 = 1 + iS 2 . 

4.73. 8 2 / in terms of /" and its differences 

We will use some of these relations between different ial operators and 
finite difference operators to express 8-/u * n terms of/ 0 and the central 
differences of f" at x = x 0 . A first approximation is 

8 2 / 0 = ( 8 .r) 2 /o = (&r)*Z>*/o = U%-, 
to improve on this we must find an operator <f>( 8) such that 

S 2 /o = 4>M T *f 0 - 

The operator </>( 8 ) required is therefore 

(f>( 5) = (5/f r ) - ; 

its expansion in powers of 8 is given by (4.47) above, arid substitution 
in (4.53) gives 

s 2 /o = otSx)n 

(4.54) 

(compare formula (4.19) and its derivation in §4.5). 


(4.51) 

(4.52) 


(4.53) 
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4.74. S/ t symmetrically in terms of /' and its differences at x 0 
and x 1 

By definition, S/j = f x — f 0 = (E—l)f 0 , and to a first approximation 
S/i = = h(8x)(E+l)Df 0 = l(E+l)Uf 0 ; (4.55) 

we want to obtain a more general relation to which this is the first 
approximation. 

By a formula symmetrical in/'and its derivatives at the two ends of 
the tabular interval is meant one in which the coefficient a n of each 
S"/l is the same as that of the corresponding B n f' 0 , so that these terms 
together give a contribution a n (8 n f 0 + S"/i) = a n 8"(E+ l)Df 0 , as the 

terms with n■ = 0 do in the first approximation (4.55). Hence we want 
a relation of the form 


5 Si = W-\)f 0 = lJ(&)(E+l)Uf 0 ; (4.56) 

to satisfy (4.5G), <^>(S) must be given by 

+ i5) = WTTju- < 4 - 57 > 

This can bc expressed in terms of U by substituting E = e u ; this gives 

<f>( 8) = (tanh \U)fhU. 

Alternate ely, it follows from (4..’14) that (4.57) can be written 

6(8) = S 'mU 

‘or which the expansion in powers of 8 is given by (4.51). Hence the 
required formula is 

- 5(W.^/',-,W.+8r 1 )+Aj, ( sv;+8Vi)- 

- *JSi,(S*/o+S 8 /i)] + 0( Sx)>. (4.58) 
■Hr; is an integration formula, for if/'(.r) is given as a function of* it 

’ than ^ *-/<•*•)• i8 J/V) dx, to be evaluated in terms of 

<hf values of/ (.r) (see Chapter I). 

4.75. p5/ 0 in terms of f and its differences at x = x 
In tins case we want to find an operator 6(8) such that 

/*Vo = (3.r)^(8)Z>/ 0 . 

'I'lie appropria.t.' rf{$) is 

/(8) - pS f — (sinh {7)/t/ 
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of which the expansion in powers of 8 is given by (4.52). Hence 

Mo = i(A-f-i) = (*x)U' 0 +tf 2 fo- 

(4.59) 

(compare §4.5, formula (4.21)). This also is an integration formula, 
relating the change of / in an interval 2S.r of x to the behaviour of its 
derivative in the neighbourhood of that interval. 


V 

INTERPOLATION 


5.1. Linear and non-linear interpolation 

Given a table of values of a function f(x) at a set of tabular values of 
x, usually, but not necessarily, equally spaced, we may require to deter¬ 
mine either the value of f(x) at an intermediate value of x, or the value of 
x for which f (x) has some specified value. The process for finding a 
result of this kind is - called ‘interpolation’, and, when it is necessary to 
distinguish between them, the former is called ‘direct’ and the latter 
inverse interpolation. The distinction is not usually significant unless 
the tabular values of z are equally spaced; this case, however, is much the 
most usual. 

By linear interpolation is meant interpolation using the approxima¬ 
tion in which, for 0 < 0 < 1, we take 


f ( x o “b ^ &c) — /o +0 8/j; (5.1) 

expressed graphically, this is interpolation along the chord joining the 
points (.r 0 , f 0 ) and (*„ /,). This process is valid so long as the tabular 
values of .t are spaced closely enough; we will obtain later (§5.12) a 
quantitative criterion of what is ‘closely enough’ in this context. ‘Non¬ 
linear interpolation’ is interpolation in some form which takes account 
of the departure of the (.r, /) curve from the chord between the points 
corresponding to neighbouring tabular values. 

There are two kinds of tables; first, those in which interpolation is 
re. pi i red frequently enough to justify the use of intervals of the argument 
small enough for linear interpolation to be adequate. Secondly, there 
are those m which interpolation will only be occasional, not frequent 
•■m.uj.i to justify the calculation and printing at small enough intervals 
lor ,,nea r interpolation to be applicable. 

In the latter ease, non linear interpolation is necessary. But if non- 
inear interpolation were generally recognized as a standard process, 
the bulk of tables could be very greatly reduced. For example, a table 

° h ,' C de " maIs at int orvals »f 10° reads as given on p. 61. 

\ Ska ‘ 8ee ia,er ,hat the formulae required for carrying out non-linear 
mterpolatmn in this table are comparatively simple. We shall also see 
that for linear interpolation wo require |8=/| to be not greater than 2, 

obtain a“! hi vTr““ nU '” b ° r ° f ° ntr,cs are re 1 uired in order to 

obtain a table in which linear interpolation can be carried out 
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/(*) = 


X 

sin x 

8/ 

8 J / 

sy 

sy 

0° 

0 

17365 

0 

-528 

0 

10° 

•17365 

16837 

- 528 

-511 

+ 17 

20° 

•34202 

15798 

-1039 

-480 

31 

30° 

•50000 

14279 

-1519 

-435 

45 

40° 

•64279 

12325 

-1954 

-372 

63 

50° 

•76604 

9999 

-2326 

-307 

65 

60° 

•86603 

7366 

-2633 

-221 

86 

70° 

•93969 

4512 

-2854 

-139 

82 

80° 

•98481 

1519 

-2993 

- 45 

94 

0 

o 

05 

1-00000 


-3038 


90 


The reduction in bulk achieved by the use of a large interval and non¬ 
linear interpolation is not important in the case of functions of a single 
real variable, but becomes important in connexion with functions of 
two variables (or of a complex variable), or functions of a variable and 
one or more parameters such as the Bessel functions J n {x) and the 
Whittaker functions W k m {x). 

5.11. Linear interpolation 

The simplest form of interpolation is linear interpolation, or inter¬ 
polation by proportional parts, for which the interpolation formula is 
(5.1) above. 

In carrying out linear interpolation on a machine, there is a precaution 
against mistakes which should always be observed. Suppose first that 
is positive. Having cleared the accumulator, set / 0 , add it into the 
accumulator, and clear the multiplier register. Then set 8/j. add it in, 
and verify that the content of the accumulator is now f v This checks that 
the right values of f 0 and 5/, have been taken. If 0 has m dec muds, the 
accumulator should first be shifted m places right, and f 0 and df { then 
set on the extreme right of the setting levers or keyboard. 

For direct interpolation, 08/* is added to / 0 ; if 0 is greater than j. t ie 
addition of 8/j to f Q to check can be taken as the first p u * l,s 111,1 
plication; if 6 is less than 8/* should be subtracted from f, to restore 

fo before doing the multiplication. 

For inverse interpolation, the given value of/is built up in t ie acuinm 
lator, and the fraction 0 of the interval length required to give this value 

°f/is read on the multiplier register. 
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If S/j is negative, |S/ t | should be set, and operations of addition and 
subtraction are interchanged; otherwise the procedure is the same. 

In some tables, particularly elementary ones, a sequence of function 
values is given on a single line (for example, log 1-00 to 1-09 on a line of 
a four-figure table of logarithms) with proportional parts of the mean 
first difference at the end of the line. Use of these proportional parts 
of the mean difference does not usually give the best interpolated 
value, and should not be used indiscriminately except in contexts in 
which an error of 2 or 3 units in the last figure is unimportant. The 
following example is taken from a table of logarithms to five places in 
which, for x = 1-0 to 2-0, different sets of proportional parts of mean 
differences are given for every Jive entries: 


X 

logx 


105 

02119 


100 

02531 


1 07 
02938 


108 

03342 


109 

03743 


1 2 3 . . 5 ... 9 

40 81 121 . . 202 . . 364 


1 he last five columns are the proportional parts of the mean differ¬ 
ence. Using the actual difference between the first two entries, we get 
log 1-055 = 0-02325, whereas using the proportional parts of mean differ¬ 
ences we get 0-02321, a difference of four units in the fifth decimal. 

For the best linear interpolation, proportional parts should be taken 
of the actual difference between successive tabular values. Tables of 

proportional parts for this purpose are given in most good modern books 
of tables. 

If several functions are tabulated in parallel columns, at such an 
interval that linear interpolation can be used on each of them, then 
linear interpolation can be used between two columns. For example 
a table of sin* and cos* against *, in parallel columns, is also a table’ 
" ( 1 ~V )* against //, and can be used as such without reference to the 
eoluma at all. Since the values of both functions f(x) and cj(x) are 
subject to rounding error, the possible error in the interpolated value is 
.ather greater than d/(*) were tabulated at exact values of </(*). 

5.2. Non-linear interpolation 

T'frT 8 UO " linear intcr I ,olat ion> it will be supposed for the 
Intern I ' ' ''*'“** ° f ,hc ar g umcnt «« equally spaced, 

considered hi ^ ^ V “ ,U “ ° f th * will be 


5.21. Half-way interpolation 

One particular case ..f non-linear interpolation is so much simpler 
than the general ease, and so useful, that it will be considered separately 
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first. This is interpolation for a value of x half-way between tabular 
values. 

To get a formula for this, we want to express f(x 0 -\- \bx), which can 
be expressed as E*f 0 , symmetrically in f 0 , f x and the differences of / at 
x 0 and x v Now/(ar 0 -f- £8x) = / 0 , so we want to find an operator 0(8) 

such that Elf 0 = W)l( l + £)/„. 

This operator is given by 

0(8) = 2 E*/{l + E) = 1/cosh W = l/(l + ]8 2 )‘, (5.2) 

so that 

h =/(*o +m = (l-H^)-»[l(/o+/i)] 

= i|/o+/i-l(8 2 /o+S 2 /i) + illi(SVo+8 4 /i)- 

It will be noted that the operator 0(8) given by (5.2) is the operator 
inverse to the averaging operator /z. Indeed, the relation (5.3) could be 
obtained as follows. The definition of the operator /z is /z/ t = i(/ 0 +/i). 
and it follows that the inverse operator /z _1 is an operator such that 

h = •1m- , (/o+/ 1 )- < 3 - 4 > 

But /j = f{x Q -\-\hx) and /z 2 = l-f-JS 2 , so that (5.4) is just (5.2) in a 

different form. 

Formula (5.3), perhaps taken to higher orders of differences, is useful 
in a preliminary breaking down of the interval of a table of a function 
evaluated at a large interval, before carrying out a subtabulation. 1 he 
coefficients are easy to calculate and to check if more arc required than 
are given in (5.3). If ( — )'«, is the coefficient of (8 2 '/ 0 -{-S 2 '/,) in the square 

bracket in (5.3) then = {2 j+l)/8(j+l) 

and the coefficients can most conveniently be calculated b\ conlimud 
multiplication by the successive ratios (5.5). A check is gi\en b\ the 
relations £ a } = 2/V3 = 1- 15470. £ 2 * a j = v “ “ I ' 414 ' 1 - 

It is interesting to examine the* result of applying (5.3) to a "* S|M ' 

at a large interval such as GO 3 or 90 '. The rut io (5.5) tends to tla- value , lor huge.,. 
Hence provided |8 2 ' + 7/8 2 '/| < 4 for large j, the intinite series ot Inch (3-1) 

the first fow terms formally converges. 

Now if J( X ) = D coh(x + P). then 5 -fj = -2( 1 -cosSs)f t , s<» that n 
|S 2/+2 //S 2> /| = 2. Thus by use of the series (5.3) \ve ear. mtei point- eos.i 
not only approximately but, by taking enough terms, to an\ ate ma* \ 11 1 

from the tabular values 

\tt 77 

0-1 0 1 
1 0-19 


I 8.r 4^7. 

>s x ami sin s. 


x 

cosx 
8in x 


0 

1 

0 


and the condition of periodicity. 
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It is even possible to interpolate accurately from a table at intervals of §7r: 


X 

0 

$7T 


2 TT 

cosx 

i 

-1 

-* 

1 

8 in x 

0 

*V3 

-*V3 

0 


extended by using the condition of periodicity. 

5.22. Newton’s forward-difference formula 

Of the formulae for non-linear interpolation for a general value of 
the fraction 6 of the interval length, the simplest to derive is one in terms 
of forward differences. Its practical value is, however, limited. 

Taylor’s series can be written, in terms of operators, 

f(x 0 +OSx) = e^/o = E% 

Also E = 1-f-A, and expansion of (l-f A) 0 by the binomial theorem gives 

f(x 0 +9Sx) = [l+0A+Ie(9-l)4“+...]/o 

= /o+9A/o + ^»(«-1)A 2 /o + ^0(»-1)(«-2)A 3 /o+- (5-6; 

which is usually known as Newton’s formula. It uses values of the 
differences on an inclined line in a difference table: 

x / 

*0 /o 



It is unsatisfactory if differences beyond the second have to be taken 
into account, as the differences of a function /depend primarily on the 
behaviour of the function in the neighbourhood of the value of x on 
which they are centred, so that the higher-order differences involved in 
this formula are less and less closely related to the behaviour of/in the 
interval in which interpolation is being carried out. 

Its practical use is restricted to interpolation near the boundaries of 
a table, and this is rare because unless / or one of its derivatives has a 
singularity at x — x 0 , there should usually be little difficulty in extending 
the table backwards a few intervals from x = ,r 0 , whereas if the boundary 
oi the table results from / being infinite at x =■ x Q (for example f(x) = 
cot x at x — 0) or undefined for x < x 0 (for example f(x) = at x = 0), 
this situation is usually associated with an infinite derivative f'(x 0 ), in 
which case the laylor series expansion on which Newton’s formula is 
based is invalid. / 
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There are various other interpolation formulae, which can all be 
derived from Newton’s by substitution for the forward differences 
A n /o in terms of differences more representative of the behaviour/!*) 
in the interval through which the interpolation is being carried out It 
is difficult, however, to obtain the form of the general term by such a 
derivation, and it is better to derive these other interpolation formulae 
independently. Of the various formulae Comrie writes! ‘only three are 
found in good modern practice, namely those associated with the names 
of Bessel and Everett, each of which is a simple transformation of the 
other, and that of Lagrange’. The present treatment will be restricted 

to these three. . 

From Newton’s (or Bessel’s) formula it is possible to deduce the con¬ 
ditions in which linear interpolation gives a sufficiently accurate result. 
The greatest numerical value of the coefficient of the second difference 
in formula (5.6) is J. It is best to keep the contribution from this term 
to the interpolated value less than 0-3 in the last figure; if it were 
greater it should be included as it might affect the rounding off of he 
final result. Hence linear interpolation should not be used > f ^ond 
differences are greater than 2 unless errors up to - units in 

place of the interpolated value can be tolerated 

Occasionally the contribution from the second differences to the inter - 
polated value is negligible when those from higher orders differences 
are not; an example is provided by the function *(**- D(» “*> 
at unit intervals of, and interpolated between , = ° ^ 
this situation it is only necessary to sec that not on } 
ences used in the interpolation formula, but also a number of neighbor,. 

ing values, are not greater than 2 in the last hgure. 

SSK!«SS± 

for circular functions of a numerical \aname ,3 

will be given here for completeness. numerical 

For the purposes of this section, let u stand for an ordinary numeric 

variable, and let % __ 2s inh \u and y = cosh0 m. 

t Chambers' a 6-Figure Tables, vol. 2 (1949), J. CJ. L. Mich-1. 

t Tho treatment of this and the following bect.on follow 

Journ. Inst, o] Actuaries. 72 (1940), 4/0. <. (Mucinillun, 2nd «d. 1920) § t.H. 

§ e.g. T. J. I’A. Bromwich, Theory oj Infinite Senes (Mucn 
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Consider first the expansion of y as a power series in z. We shall obtain 
this by forming the differential equation for y in terms of z as independent 
variable, then differentiating n times and putting z = 0; this will give 
recurrence relations for the derivatives d n y/dz n at z = 0, from which 
their values, and so the required series, can be written down. 

Since y = cosh fill, it satisfies the equation 

8 = ( 5 - 8 ) 

and since z = 2 sinh \u, it follows that 



(5.9) 


so that 


8 = ^osh [<cosh iu) d £ 


On differentiating this out, substituting for sinh 4?* from (5.7) and for 
dujdz from (5.9), we obtain 




(5.10) 


and then, on differentiating n times with respect to z and putting z = 0, 

yn + 2, (0) = (5.11) 

Also, for small u, z = u-j-0(u 3 ), and so 


y = l+0(«2) = 1 -f 0(z 2 ) 

and hence *(0) = 1, y' { 0) -= 0. Hence, from (5.11), for the odd deriva¬ 
tives 

y 2n+1, (0) = 0 

and for the even derivatives 




S 2 . 


//"■(O) = W 2 - 1), y"i(0) = PiP-lKP-i)..., 

and m general, in terms of the functions 0„, introduced in §4.41, 




iL’iici 


(5.12) 


.7 = cosh flu = 1 + fi | 0W/Os*"*7(2« + 2)!. 
I inferential ion with respect to z then gives 


du 


But from (5.9) 


,5(smh p ’„) = 0; £ K-n(P)z*'-H2n + l)l 


zdz.dit 


(2sinh hi) cosh hi = s inh u; 
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hence 


sinh flu 
sinhw 


n 


(5.13) 


= (3 l+ij(/3 , - 1 ) 2 , + i(^- 1 )(^-4) 2 '*+...]. (5.14) 


To obtain corresponding expressions for sinh fiu and cosh/?u/cosh ^w, take 
z = 2 sinh \u as before, and y = sinh/?u. This also satisfies equation (5.8), and the 
abovo argument.applies as far as the recurrence relation (5.11). Now, however, 
y = fiz + 0(z 3 ) for small 2 , so that y( 0) = 0, j/'( 0) = /?. Hence y ,2n, (0) = 0, and 

2/(0) = y”( 0) = y r (0) = /9(/3 2 -i)(/3*—f). 

and in general, in terms of the functions ip m of § 4.41 (p. 41), 

2/' 2n+,, (0) = 

(compare (5.12) for the expansion of coshjSu). Hence 

y = sinh )9u = )?I0 2ri ^)= 2n+1 /(2M+l)! (5.15) 

n 




(6.16) 


The expansion of (cosh/Ju)/(cosh %u) can be obtained by differentiating (5.16) 
with respect to z. On the left-hand side this gives /3(eosh f3u) (dujdz). But from 
(5.9) this is just /?(cosh/3u)/(cosh Ju). Honeo 




(5.17) 


5.4. Everett’s interpolation formula 

The simplest central-difference interpolation formula to obtain is that 
known as Everett’s. This expresses the interpolated value of /in terms 
of the values of/ and of its eren-order differences only, at the beginning 
and end of the interval in which the interpolation is being carried out; 
that is, it is of the form 


f(x u + 08z) = (l-<O/o+0/ l + ^WV«+ W)SVi + 

+ K(d) 87o+£‘i v (0) S7i + - • (5.18) 

The coefficients in this interpolation formula arc usually known as 
‘Everett interpolation coefficients’; they arc functions of the fraction 0 
of the interval length hx for which the interpolation is being carried out. 

To obtain a formula of this kind we must find operators / 0 (8), </>j(S) 
which involve only even powers of o and which are such that 

f(x o +08x) = (5. 19) 

Now f(x 0 -\-6dx) = e 0u f„, and = e u f 0 , so (5.19), expressed as a rela¬ 
tion between operators, becomes 

e eu = &,(»)+*.<»)*''• 
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Since </> 0 (8), <f>i(B) are to be even functions of 8, and so of U, it follows 
that they do not change on replacing U by — U\ hence 

^e-^ = ^ 0 (8)+^(8)e-^ 

and solution for <f> 0 {S), <^(8) fhen gives 

^(8) = (e eu —e- eu )/(e u —c~ u ) = smh0t7/sinh U 
<£ 0 (8) = sinh(l — 0)E7/sinh U = sinh#t7/sinh U, 

where, for convenience later, B has been written for (1—6). 

We require expressions for these operators in terms of 8. These could 
be obtained by expanding in powers of U and then substituting for U 
in terms of 8 from formulae (4.43) to (4.45); but the form of the general 
term is most easily obtained by the formal substitution of U for u, 
8 for z, and 6 or B = 1-0 for p in (5.13) or (5.14). This, followed by 
substitution of the results into (5.19), gives 


Jto+etz) = 2 1 - S2,l /i] ( 5 - 20 > 


n 


= B 


+ 


/o + i tf 2 - 1) ■5 2 /o + i (P— 1)(? 2 -4) SV.+-] 

+ «[/,+ 3 , (« 2 - l)»7i + i (9 s -1)<0 2 ~4) SV.+-] (S- 21 ) 

which is Everett's interpolation formula. 

Comparison of (5.20) with (5.18) gives the following general expres¬ 
sions for the coefficients in (5.IS): 

£l n (B) ^i(l-6)/(2n+l)\\ 0) 


5.41. Bessel’s interpolation formula 

Bessel’s interpolation formula expresses the interpolated value 
/(.r 0 -f 08.r) in terms of mean differences of even-order p8 2, ‘/i and odd-order 
differences 8 2 ,,t1 /j. centred on the middle of the interval in which inter¬ 
polation is being carried out. For practical work it is most convenient 
to have the contribution from the even-order differences expressed in 
terms of the sum (S 2n /o+8 2 "/ 1 ) of the values at the beginning and end of 
the interval. Thus this formula is of the general fonnf 

f[x o +0te) = Wo+LYHO-hW i +B u m* i Jo+* t fi)+ 

-I- + X f l ) + B*(0)&f i + .... (5.23) 


t In tin-. - formula. /'.“>*( tf) is writ ton for the* coefTiciont of (5 3 / 0 -*- S 2 /,). not for tho eo- 
efliriorit of and >inu!urly tor the higher even orders of differences. This usage 

follows that adopted by C'oinrio [Chambers'* 6-Figure Tables, llMth vol. 2). In some 
earlier work and tabulation of coefficients in Bessel s formulae B or BU has been used 
for tho coefficient of ,i5 2 /j. 
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The coefficients B n (6) of successive orders of differences in this 
formula are called ‘Bessel interpolation coefficients’, or simply ‘Bessel 
coefficients’ when there is no danger of confusion with the other meaning 
of this term. The first two terms give the value/ o H-08/j obtained by 
linear interpolation, expressed in a form consistent with the other terms 
of the series. 

A formula of this kind can be derived quite easily from Everett s 
formula (5.18). Consider the pairs of terms involving 8 2 "/o and 8 2 "/i 
in Everett’s formula. These can be written 


Ei"(e)h 2 ’'f 0 +Ei"(6)V''f l 

which is of the form of the contributions from b 2 "f and S 2,,+1 / in Bessel's 
formula. Comparison with (5.23) and use of the formulae (5.22) for the 
Everett coefficients gives 

B 2n (0) = \[El n (d) + El"(8)] 


1 


2 (2n + l)! 




and 


B 2 "+'(0) = l[El"(B)-Ei"(e)] = * ( 2^py)1 1 - 0)J- 


and on substitution from (4.13), (4.14) these become 


= i - 1 


2 (2/i)! 

1 


(5.24) 


B 2n ^(d) = 

v ’ (2« +1)! 


The first few functions B"'(0) are 

B"(0) = 0(6— l)/2.2! B l[l (0) = 0(0—\)(6—\)!'M 

B ly (0) = (6-\-\)0(0—1)(0 —2)/2.4! 

B*(d) = (0+ 1)0(0- \)(d-\)(0-2)IZ\ 


Bessel’s formula can alternatively bo derived directly without 
formula; the following is a summary of this derivation. 

Expressed in terms of operators, formula (5.23) can be \witt< n 

!■:% = ^(8)(/^ D/,+0)^/o 


using Everett's 


where <^(8) is an even function of b and <f> 2 {&) an G< ^ hint tion 
<f>i(b) and <f> 2 {b) must satisfy 


Thus the operators 


E°~l = ^,(8)(E» I L’-») 4 <^(8), 
e ,e ^ ^,(8). (2cosh $*•') + <^>*(8). 


that in 
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But c'M)£ = cosh(9-$)U+smh(d-±)U, 

of which the first term is an even function of U and so of 8, and the second is an 
odd function. Hence wo obtain a formula of the kind sought by taking 

&(8) = cosh(9-i)U/2coshW, <f> 2 {8) = sinh(9~l)U. 

The expansions of these in powers of 8 = 2sinh$C7 can be written down from 
(5. 1 6), (5. 17) by making the formal substitutions of U for u, 8 for z, and (9— \) for (3. 


5.42. Use of Bessel’s and Everett’s formulae 
Bessel’s formula to second differences, namely, 

f(x 0 +98x) = / o +08/ t +^)(8Vo+8 2 /i), (5.25) 

or to third differences, with second differences modified as explained 

below, is generally the most useful formula for non-linear interpolation, 

unless so large a number of figures is required, or the spacing 8x is so 

large, that fourth and perhaps higher-order differences have to be taken 

into account. Then Everett’s formula is probably more convenient, 

especially when using tables in which only differences of even order are 
tabulated. 

The coefficient B"(9) is always negative. A critical table to four 
decimals is given in Comrie and Milne-Thoinpson’s Standard 4-Figure 
Tables and one to five decimals in Interpolation and Allied Tables, 
where B'"{9) is also tabulated; B li [9) and B M (6) arc also tabulated in 
Chambers's G- Figure. Tables, vol. 2 (1949). For other tables of coefficients 
m this (and other) interpolation formulae, reference should be made to 
the Index of Mathematieal Tables. 


In Bessel's interpolation formula, the coefficients of the odd-order 
di treronees are all zero at 9 = h as well as at 9 = 0 and 1; this is an ad van- 
tag.' over most other interpolation formulae which involve all orders 
<*! differences. The greatest value of |tf»(f?)l is about 0-008, so the 
contnlmti«in from of to the interpolated value is less than 0-5 in the 
significant figure if jS 3 /| is less than 60. 

j-urther. the contribution from the second and fourth differences 

^ , ^)[(8Vo+8V,)+i , ^+ 1)(^-2)(84/ 0 +SY i) ] 

Tu , U9+ I’-u - ?! ‘!° e8 n °‘ Vary greatly ° ver the ran 8 e of e, from 
. ! ° 1 V, ' r " h “' h thla formula " il1 be <>sed; its maximum value is 0-1875 

V lue is 0 T r rrz ,h T °' 180 0Ver half this of 9; its smallest 

s e o » n = , '• "' 1,CrC B ‘ W ’ by " hich » multiplied. 

_ «ro. Hence a good approximation to the contribution from SV to 

the m erpdated value can be made by subtracting a constant multiple 

of 5/from each 5-/, and applying Bessel's formula, correct to second or 
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third differences only, with the second differences so modified. If we 

write h* m fj = Vf,-CVJi (5.26) 

and use .B ,I (0)(8£ ll /o+Sm/i) suc h a formula, the residual contribution 
from S 4 /is ^ ( 0_ 1) [C+ i ^+ 1 ) ( ^-2)]^Sy*. (5.27) 

The best value of C is that which makes the extreme values of the 
coefficient here equal and opposite, and is C = 0-184; the greatest value 
of the coefficient of 8 4 / in (5.27) is then 0-00045, whereas the greatest 
value of \B iv {0)\ is 0-0117. The residual contribution from 8 4 /is less 
than 0-5 in the least significant figure if 8 4 / is less than 1100. 

Quantities 8 2 ,/ ; given by (5.26) with C = 0-184 are called ‘modified 
second differences’ and this inclusion of a constant multiple of the 
fourth differences in modified second differences is called ‘throwback’ 
of the fourth differences to the second. It is due to L. J. Comrie, and is 
a valuable device for simplifying practical interpolation, particularly 
inverse interpolation and subtabulation. 

In Everett’s formula El n , the coefficient of 8 2n f 0 is the same function 
of (1—0) as E\ n is of 0, so that in tables of interpolation coefficients the 
number of separate functions which have be to tabulated for Everett’s 
formula is only about half as many as for formulae involving all orders of 
differences. Also in tables of the function /to be interpolated, only even- 
order differences need be given. Tables of Everett coefficients are given 
in Interpolation and Allied Tables and in Chambers's 6-Figure Tables, 
vol. 2 (1949). 

The ‘throwback’ can be used with Everett’s formula as with Bessel's. 
The contribution from S 2 ^ and SVj together in Everett’s formula is 

W 2 — i)[s 2 /i+^ 2 -4)5 4 /,]; 

the coefficient — } u (6 2 -*) varies from 0-15 to 0-20 over the range 0 = 0 
to 1, and is multiplied by a zero factor at both ends of the range. If 
the same modified second differences are used, namely, 

8 2 ,/ = 8 2 /—0-184S 4 /, 

the residual contribution from 8 4 /j is 

£0(0 2 -1 )[0-184 + 1 6 ( 0 2 - 4)] 8 4 /i I 

the greatest value of this coefficient is about 0-0008, so that this con¬ 
tribution is less than 0-3 in the last figure if S 4 /, is less than about 400. 
Similarly for the contribution from S 4 / 0 . 
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If fourth differences are too large to be treated by means of the throw¬ 
back, Everett’s formula can be taken as far as the 8 4 / terms, and the 
sixth differences thrown back to the fourth differences.! If eighth 
differences are appreciable, very effective use can be made of a joint 
throwback of the sixth and eighth differences to the second and fourth 
differences.! 

5.43. Practical details in non-linear interpolation 

In using Bessel’s or Everett’s formulae, values of the coefficients can 
either be calculated as required or taken from tables. In the latter case 
the interpolation will have to be done in two stages if the number of 
decimals in 6 is greater than that in the argument of the tables of inter¬ 
polation coefficients. One method of dealing with this situation is to 
carry out a subsidiary interpolation in the tables of interpolation 
coefficients themselves. But it is generally better to carry out a small 
subtabulat ion of the function / (.r) using only tabular values of the inter¬ 
polation coefficients. For example, if f(x) is tabulated at intervals 
3.r = 01 and its value is wanted for x = 0-854377, and available tables 
of the interpolation coefficients have the argument d at intervals S 9 — 
0 001, the values of f(x) for a- = 0-8541(-0001)0-8545 can be obtained 
without interpolation in the tables of the interpolation coefficients, and 
interpolation in this small table of f{x) will then give the result required; 
linear interpolation will often be adequate at this stage. 

In carrying out a non-linear interpolation, it is advisable to carry 
one guarding figure to avoid accumulation of rounding errors from the 
various contributions to the interpolated value. For a similar reason, 
a guarding figure should be kept in the subtabulation mentioned in the 
previous paragraph. Also it is advisable to retain contributions greater 
than 0-2 in 1 he least significant digit from the higher orders of differences. 

On this basis: 

In Bessel's formula, with throwback of fourth differences to second: 

8 3 / can be neglected if less than 15 
o 4 / can be neglected if less than 500. 
hi Everett s formula, with throwback of fourth differences to second: 

(V 1 / can be neglected if less than 250. 

t For thoso ami other dovolopmonta of tho idea of tho throwback, soo Chambers's 
6 Figure Tables, voh 2 (1'J IO), j>. 533. 
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Examples-. 

(a) Given the following values, to find/(§): 

SV 


X 

/(*) 

0-60 

1-66667 

•62 

•61290 

•64 

•56250 

•66 

•51515 

•68 

•47059 

•70 

•42857 

0-72 

1-38889 

= 0-02, x 

= 5 = 0-66 


8 / 

— OOH 

-5040 

-4735 

-4456 

-4202 

-3968 


h'J (8 2 / 0 + Sy,) 


337 

305 

279 

254 

234 


-32 

-26 

-25 

-20 


+ 533 


difference is just worth tak.ng into account, — ‘ / V n " value of BH6) is 
bution is negligible, even without using the throwback.of * <*> 
\8(Q— 1) = — A. and that of B lll (d) is 19(8 — 1)(0 £) + I#! ' 


f 0 =z 1-51515 
Qhfi = -0-01485, 

BH0)\hy o-i Sy,l = -0 00029. 

= — 0 + 0000 , 

1-49999,, 
or rounded off, 1-50000. 

i n ciTiv this is a convenient convention. 

The guarding figure is written here m >■ ■ ’ ' contrib u«ion in the form 

Notes: (i) The point of expressing the second -d.tlerc no , 0 lf tho 

WMB'/.-W,). rather than { 2 fi»( 0 )>p 8 y,, is^elear ^fro ^ ^ wouM eithor 

quantity + is odd, then in div ■< >ng . nvoid(I / 1 bv incorporating the 

have to round off or keep an extra figuu, cnmntitv is multiplied. 

division by 2 in the factor **•(«> = 1 * ''ZZZZLd off to five 

(ii) The function /(x) here is 1/x. the tubular values being .ounded 

decimals, so that the correct value of/(5) i* 1 '* 1 xuct * V 

a * s i ft a 


(8) 

Given tho following vn 

lues, to 

find 




X 

/(*) 


sy 

sy 

sy 

— 0 1848'/ 

hlJ 

0-60 

2 00000 

-18182 






•55 

1-81818 


3031 





•00 

•66667 

- 15151 

2330 

— <01 

203 

--37 

2293 

•65 

•53846 

-12821 

1832 

— 498 

131 

— 24 

1808 

•70 

•42857 

- 10989 

1465 

— 30 1 

93 

-17 

1448 



- 9524 


— 2 <4 

63 

- 12 

1 179 

•75 

•33333 

- 8333 

1191 

-21 I 




•80 

•25000 

- 7353 

980 





0-85 

117647 








3256 


76 


INTERPOLATION 


(iii) At least one of the coefficients L s (8) is greater than 0-5; if there¬ 
fore an interpolation is required for a value of 9 which is not a 
tabular value in the table of Lagrangian interpolation coefficients, 
interpolation in these tables is required to the same number of 
significant figures as that required for the interpolation of/itself; 

(iv) Use of Lagrangian formulae does not lend itself to an easy process 
for inverse interpolation. 

To these mav be added: 

* 


(v) They do not provide the facilities for improving the accuracy of 
interpolation without complicating the formulae, such as are 
provided bv interpolation formulae in terms of differences by 
use of the ‘throwback’. The Lagrangian formulae are based on 
the approximation to the function to be interpolated by a poly¬ 
nomial of the wth degree through (a-f 1) points. But consider 
the significance of the use of the throwback from fourth to second 
differences and subsequent use of Kverett’s formulae to second 
differences. The fact that this modification of the second differ¬ 
ences improves the accuracy of the interpolation (and moreover 
by a factor of about 10. not only by a small amount) means that 
for interpolation between /„ and /, the best cubic is not the cubic 
through /„. /,, fo which is the one used in the four-point 
Lagrange formula. 

ula of Lagrangian type corresponding 
to the Evcreft formula to second differences, used with modified 
second differences; but this is no simpler than a six-point Lagran- 
gian formula based on the use of a quintic polynomial. The 
corresponding formula using differences is Everett’s to fourth 
differences, and if in this the joint throwback of sixth and eighth 
differences to second and fourth is used, a very substantial 
improvement in the accuracy, compared with that of a six-point 
Lagrangian interpolation, is achieved. 

Comrio-s commentt is that he ‘has to admit that his experience has 

not I)im P art,nI to bljnd Lagrangian interpolation, except when 
special circumstances point very definitely to it’. 


n.ol. Special interpolation methods for particular functions 

For some particular functions, special interpolation methods may be 
convenient- than the use of the Bessel or Everett formulae. For 

1 Chambers s 6-Figur* Tables, vol. 2 (I'M!*). Introduction, p. xxx. 
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example, for the exponential function it may be most convenient to use 
the addition formula eZ+v _ t x tV 

and carry out interpolation by a multiplication or succession of multi¬ 
plications. If, for example, e x is tabulated at intervals of 0 01 in x, an auxi¬ 
liary table for x = 0(0 0001)0 01 would enable values to be obtained for 
four-decimal values of x in the range of the main table by a single multi¬ 
plication; alternatively a set of auxiliary tables for x = [0(0-l)l]x 10 
for n from 2 to 5, say, would enable values of e' for six-decimal values 
of x to be obtained with not more than four multiplications. 

If the function y to be interpolated satisfies a simple differential equa¬ 
tion such as y" = xy, formulae for successive derivatives of y, obtained 
by successive differentiation of the differential equation, may be simple 
enough to be used for the numerical evaluation of these derivatives. 
Then Taylor’s series can be used for interpolation between tabular 
values, f{x 0 + 6Sx) being calculated from as many terms of the series 

/(Xo+ 0 8 *) = f'+e(&x)f'+±eHSx)%+l<?i*x)y°+-- 

as are appreciable. A good check is provided by evaluation of the 
alternative expansion, in terms of/and its derivatives at x = x„ namely. 

/(x o +0x) = /(Xj-flSx) (Where 0 = \-8) 

= +1 onsmi -1 *i**>vr+~ • 

The convenient quantities to tabulate for interpolation purposes are 
not the derivatives/'*> but the quantities [<S*)*/*!]/"". sometimes called 
■reduced derivatives’. This method of interpolation is particularly 
convenient in the case of functions which have been evaluated by- 
integration of the appropriate differential equation by the Taylor scries 
method (§ 7.4), since the reduced derivatives are evaluated in the course 
of this calculation, and so are available for interpolation purposes with¬ 
out any further work. An example is provided by the tablesl of the 
function usually written Bi<*>. which is one solution of the equation 

y" = xy. 

5.6. Subtabulation 

Subtabulation is a special case of interpolation, of which the purpose 
is to take a function f(x) at tabular interval &x and construct from it a 
table at a smaller tabular interval qix; in practice q is usually J, s , or M . 


table at a smaller 

British Association 


Mathematical Tables, Part-volume B. The Airy Integral (1U46). 
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The values of the function at the large interval (8 a), between which 
interpolation is carried out, are called ‘pivotal values’ of /( x ). 

It is a valuable process when the direct calculation of /(a) is difficult 
or long, for example by summing a series of a large number of terms, 
or evaluation of a definite integral in which the integrand is a function 
of x. In such cases we want to restrict the number of values of x for 
which f(x) is calculated directly, and derive from them a table at a 
smaller interval, probably one in which second differences at most have 
to be included in interpolation, and possibly one in which linear inter¬ 
polation is adequate. There is no purpose in subtabulation if linear 
interpolation is already adequate; the purpose of subtabulation is to 
break down the tabular interval when linear interpolation is certainly 
not adequate; so it is essentiall}’ concerned with non-linear inter¬ 
polation. 

Since h n f is of order (Bx) n , the higher-order differences are very much 
reduced by even a moderate degree of subtabulation. For example, 
subtabulation to fifths reduces fourth differences by a factor of over 500 
and sixth differences by a factor of over 15,000. 

In subtabulation, a systematic set of results is required, instead of 
an isolated result as is more usual in an interpolation process. This 
suggests that a systematic procedure should be used for obtaining the 
results. 


We could interpolate a sequence of values of the function itself and 
cheek the differences; alternatively we could construct the sequence of 
second (or higher) order differences of the function at the smaller interval 
and build up from these, using the facilities for building up a function 
from its differences (the ‘National’ machine, for example, if available). 
The latter process has the advantages that (i) most of the work is done 
"ith small numbers, and (ii) a good overall check is provided by the 
reproduction of the pivotal values in the course of the summation of the 
differences of the function at the smaller interval. The former does 
not check that the correct pivotal values have been taken, and the use 

01 inc ° !Toct I >h otal valup $ may not be apparent in the differences of the 
sub tabulated function; these differences check against random errors, 
but the effect of an incorrect pivotal value is a systematic error, and may 

not be indicated by differences. This is illustrated by the followin'" 
example. 

Consider the subtabulation to tenths of sin a- from a table at intervals 
of 10b Use of an incorrect value at a- = 50° (0-76624 for 0-70604) might 
gi\e a sot <>i subtabulated values as follows: 
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X 

‘sin x' 


30° 

0-50000 


31 

1504 

1504 

32 

2992 

1488 

33 

4464 

1472 

34 

5919 

1455 

35 

7357 

1438 

36 

0-58777 

1420 

37 

0-60179 

1402 

38 

1564 

1385 

39 

2931 

1367 

40 

0-64279 

1348 

1329 

41 

5608 

1309 

42 

6917 

43 

8206 

1289 

1269 

44 

0-69475 

1248 

1226 

45 

0-70723 

46 

1949 

1203 

47 

3152 

48 

4333 

1181 

49 

5491 

1158 

50 

0-76624 

1133 



X 

sin x 


50° 

0-76624 

-16 

51 

7734 

-16 

52 

8820 

— 17 

53 

0-79881 

— 17 

54 

0-80917 

-18 

65 

1927 

— 18 

56 

2913 

— 17 

57 

3874 

-18 

58 

4809 

— 19 

59 

5719 

-19 

60 

0-86603 

-20 

61 

7461 

-20 

62 

8293 

-20 

63 

9099 

-21 

64 

0-89879 

-22 

65 

0-90631 

-23 

66 

1355 

-22 

67 

2050 

-23 

68 

2718 

— 25 

69 

3358 

-23 

70 

0-93969 


1110 

1086 

1061 

1036 

1010 

986 

961 

935 

910 

884 

858 

832 

806 

780 

752 

724 

695 

668 

640 

611 


-23 

-24 

-25 

-25 

-26 

-24 

-25 

-26 

-25 

-26 

-26 

-26 

-26 

-26 

-28 

-28 

-29 

-27 

-28 

-29 


Here the pivotal values are underlined; that at x = 50° is in error by 
20 units in the fifth decimal, the others are correct to five decimals. 
The differences of the subtabulated values are no more irregular than 
would be expected as the result of rounding errors, and certainly contain 
no suggestion of an error of 20 units. 

Certainly such an error in a pivotal value ought to be detected by 
differencing the pivotal values before beginning the subtabulation. 
But if a Lagrangian formula were used for carrying out the subtabulation, 
this step might be omitted since the differences of the pivotal values 
would not be used in the subtabulation process; if they were to be 
obtained to check the pivotal values, it would be better to use them in 
the subtabulation also. The point of this example, however, is to show 
that differencing the subtabulated values does not by itself provide an 
adequate check of the subtabulation process. 


5.61. End-figure method for subtabulation 

Comrief has given a convenient process for subtabulation, in which 
only the last digit of each interpolated value is evaluated by the use of a 
suitable interpolation formula, and the complete values are then built 
up from their differences. From the last digits in the interpolated func¬ 
tion values, only the last digits in the differences can be obtained directly; 

t L. J. Comrie, Monthly Notice#, 88 (1928), 506; Interpolation and Allied 

Talien, incorporated in Nautical Almanac, 1931. 
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but in subtabulation at a fraction q of the interval between the pivotal 
values, the nth differences of the subtabulated values are approximately 
q n times those of the pivotal values, and for some value of n these nth 
differences of the subtabulated values will vary slowly enough for their 
last digits to establish the values of the differences themselves. Suppose 
for example that, for the pivotal values, 8 2 / ; - = 610 and B 2 f j+1 = 505, 
and subtabulation is to fifths (q = |). Then the second differences of 
the subtabulated values will be approximately g 5 of those of the pivotal 
values, that is (allowing for possible effects of rounding errors) from about 
25 at the beginning of this interval to 19 at the end. Hence if the last 
digits of the second differences of the interpolated values are 

5, 3, 3, 1, 0, 

these second differences can with confidence be given the values 

25, 23, 23, 21, 20, 

and the function can then be evaluated by summation from these values 
(see § 4.46). If any mistake is made, it is shown up by the pivotal values 
not being reproduced in the summation. 

The process can be illustrated in a simple case by example ( b ) of § 5.43 
(see p. 74). If in that example the values of/(0-665) and/(0-666) had 
been evaluated in full, but only the last digits of/(0-667) and/(0-668) 
had been determined, the results (rounded off to five decimals) would 
have been: 


■x 


0*665 


0-666 0-667 0-668 


f{x) 1-50376 1-50150 



0 


Differences at the 
smaller interval 


-226 




where the dots represent digits so Jar undetermined. But, as mentioned 
on p. 74, the second differences of /at the smaller interval are about 0 8 
so each of the first differences whose last digit is 5 must be —225. The 
values of/(.r) at x — 0-667 and 0-668 c-ould then be built up from these 
differences. 


The follow ing is a more extensive example, in which it is necessary to 
go to second differences before writing down the values of a set of differ¬ 
ences from their last digits, and which also illustrates some further 
points of procedure. 
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Example: Given the following table: 


X 

/(x) 




0-310 

2-96671 

8476 



0-305 

3-05147 

8816 

340 

20 

0-300 

3-13963 

9176 

360 

21 

0-295 

3-23139 

9557 

381 

22 

0-290 

3-32696 

9960 

403 

26 

0-285 

3-42656 

10389 

429 


0-280 

3 53045 





to subtabulate to fifths (i.e. at intervals 0 001 in x) from x = 0 300 to r - 0-290. 

(Note: for this function it is convenient to take differences in the direction of 
a- decreasing, as in this table; then, apart from effects of rounding errors, they are 

Everett's formula to second differences (unmodified) is adequate in this case; 
the Everett coefficients for the points of subtabulation are: 


6 

El'(d) 

E[H6) 


0 

0 

0 


1 

6 


a 

& 


-048 

-032 


064 

■056 


- -056 
-064 


- 032 
-048 


1 

0 

0 


The second differences of the pivotal values over the range specified for the sub- 
tabulation are from 360 to 403. so for subtabulation to fifths the second differ- 
ences of the subtabulated values will be from about 14 to 16. Thus it is only 
necessary to difference the end digits to second differences in order to be sure of 
the values to be used in building up the function values. Since this budding up 
is to be done from second differences, we will need two function values from which 
to start, namely the pivotal value/(0-300) and the value/(0-299). b or the latter. 

Everett’s formula gives: 


/ (0-300) 
+ *8/(0-2995) 

+ M'(i)8 2 /(°- 300 ) 

+ £l'(i)8 J /(0-295) 
/ (0-299) 


3-13963 
1835, 

— 17 3 
- 12 * 

= 3-15768,, or, rounded off, 3-15769 


Only the rounded value is needed subsequently; guarding figures are kept in the 
contributions to it. but can be discarded in the interpolated value itself. Ouardlng 
figures, written as suffixes, will similarly be kept in the contributions to the other 

For th<» interval x = 0-300 to 0-299, the last two digit* of hf for the pivotal \ alia- 
are 76 so the first differences of the linear contrib,.tions/ 0 +«5/ 1 to the interpolated 
values 111 this interval end with 5,. The end figure <>1/(0 3(K» is 3, and successive 
additions of 6, give for the end figures of/(0-300) to/(0-29f.) inclusive the values 


'o« 


8 2 , 3 4 , 8 




9 


u 


WS 


a 
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of which the last is the end figure of the pivotal value /(0-295); this provides a 
check on the additions. For the next interval the first difference of the pivotal 
values ends with 67, so the first differences of the linear contributions to the 
interpolated values end with 1 4 , and for/(0-295) to/(0-290) inclusive are 

9o» 0«» lg, 3„ 4 # , 6„; 

the comparison of the last of these with the pivotal value/(0-290) again furnishes 
a check. 

The complete calculation can be arranged in tabular form as follows (5 is used 
for the central-difference operator at the smaller intervals): 

Last figure I 


X 

S*/o 

8*/i 

W 

(«> 

(b) 

(c) 

(d) i 

/ 

S/ 

8 */ 

sy 

8/ 

/ 

0-300' 




3 0 



= 3 0 

3 

6 



1806 

3-13903 

299 




8 , 

-17, 

- 12 , 

II 

00 

M 

9 

0 

4 

14 

1820 

5769 

298 

• 360 

381 

5, 

3« 

— 23 0 

- 21 , 

= 9, 

9 

6 

5 

16 

1835 

7589 

297 




8 , 

- 20 , 

-24, 

= 4 0 

4 

0 

6 

15 

1860 

19424 

296 




3 8 

-Us 

-18, 

= 4 0 

4 

5 

5 

15 

1865 

3-21274 

295. 




9o 



= 9„ 

9 

0 

5 

15 

1880 

3139 

294 ' 




0 4 

“18, 

— 13 0 

cT 

II 

9 

6 

6 

16 

1890 

6019 

293 




1 . 

— 24, 

- 22 , 

= 4, 

5 

1 

6 

15 

1911 

6915 

292 

■ 381 

403 

1« 

3, 

- 21 , 

— 26 0 

= 5. 

6 

7 

6 

16 

1927 

28820 

291 




4s 

- 12 , 

-19, 

= 2 , 

3 

3 

0 

16 

1943 

3-30763 

0-290. 




00 



= 60 

0 





3-32690 


(а) Linear contribution /,, + 08/j, end figure and guarding figure only. 

( б ) . (r): EV(9)b 3 f 0 and E\ '(0)8’/,. 

(d) Sum of (a), ( 6 ), (c), end figure and guarding figure only. 

The column beaded £ 8 / gives the last figure and guarding figure, as required for 
building up the linear contributions to the interpolated values in column (a). 
Columns ( 6 ) and (c) give the second-difference contributions to the interpolated 
values; these are here given in full as this makes the work easier to follow; however 
only the last full digit and a guarding figure are necessary. The sums of entries 
(a), ( 6 ), (c) with guarding figures are given in column (d) to illustrate the procedure, 
but only the rounded values of the last digit, as given in the next column, are 
required for the subsequent work. These rounded values are then differenced to 
second differences, and from the result we have already had, that the values of 8 */ 
are about 14 to 16, the complete values can now be written down from their last 
digits. Then from/(0-300),/(0-299) and these second differences, the values of/ 
can be built up. A thorough check is provided by the reproduction of all the 
pivotal values. 
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Notes: (i) In this example,/(x) is the function ((l/x) y - 1 ]/y with y = 1-4. The 
pivotal values were calculated from this defining formula; for the intermediate 
values subtabulation is a much quicker and easier process than evaluation of this 
formula. 

(ii) It is not necessary to carry out the full evaluation of/(0-299) by interpola- 
tion. The value of S/(0-2995) is approximately tV[/( 0-305)—/(0-295)]. which is 
1799 —rather larger since 8 2 / is positive—and it§ last digit is 6, which indicates the 
value 1806 with some certainty. But even if a wrong value were taken, this would 
be shown by the pivotal value/(0-295) not being reproduced; and from the amount 
of the discrepancy, the corrections te be made can easily be determined. For 
example, if 8/(0-2995) were taken as 1796 instead of 1806, each of the first five first 
differences would be in error by -10, so the discrepancy between the value of 
/(0-295) obtained by building up and the pivotal values would be — 50; this would 
indicate that each of these first differences must be increased by 10. ( This correc¬ 
tion process is not available if the subtabulated function values are built up from 

differences of higher order than the second.) 

If the second differences of the subtabulated values vary too rapidly for their 
last figures to be a certain indication of their complete values, a corresponding 
process involving building up from third or fourth differences can be used. Alter¬ 
natively, the process could be carried out with the last two digits in each sub¬ 
tabulated value instead of with the last digit only. 

Another method of subtabulation, also suggested by Comrie.t 
involves the direct calculation of second or fourth differences of the 
Bubtabulated values from formulae relating differences at interval </8x 
to differences at interval 8x. The subtabulated values are then built up 
from their differences. 

5.7. Interpolation of a function given at unequal intervals of the 
argument 

For the interpolation of a function given at unequal intervals of the 
argument, the interpolation formula usually used is that of Lagrange. 
This is based on the use of an nth degree polynomial which takes the 
given function values at (n +1) values of x. Such a formula is called an 
( n q_l)_point formula. An even number of points (odd value of n) gives 
an equal number on either side of the value of x for which the inter¬ 
polation is to be carried out. 

If the function values are f 0 , at x = x 0 , Xj.x„, not necessaiily 

equally spaced, the polynomial of lowest degree which takes these 

values is 

= f x-x y s-xa 

\x 0 —XjX 0 —x 2 x 0 — xj V*!” X 0 X l x 2 X l x ’> 


t L. J. Cornrie, Joum. Hoy. Slat. Sac.. Supplement 3 (11)36). «7 


(5.31) 
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this polynomial is of course not the function / itself, unless / is a poly¬ 
nomial of degree n or lower; it is the polynomial which coincides with/ 
at x = x 0 , x lt ...,x n . Interpolation is done by evaluating this polynomial 
for the intermediate value of x. 

If/is a polynomial of degree n or less, then F = / and the interpolation 
formula (5.31) is closely related to the expansion in partial fractions 
of//(at— x 0 )(x— aq)... (x—x n ). For if g(x) = 0 has roots x = x 0 ,x v ...x n , 
all distinct, the expansion of f(x)/g(x) in partial fractions is 


f(x) = V f(xj) 

9{x) ^g'ix^x-xj) 

and this, applied to g(x) = (;c—z 0 )(:r— xj...{x—x n ), is 

/ fo 


-f... 


(X Xq)(X X^)...(X X n ) (Xq 2 - i )( 3 "o X2)“-{Xq X n ) X Xq 

which is just another form of formula (5.31). 

Lagrange’s interpolation formula (5.31) is not restricted to functions 
given at unequal intervals of the argument. Its application when the 
intervals of a- are equal has already been mentioned in §§5.5 and 5.6, 
and its disadvantages in that context pointed out. For functions not 
tabulated at equal intervals, however, some form of it may be the only 
method available. It is then important to systematize the work of 
evaluating the polynomial, since if it is not done in a systematic way 
it is easy to make a mistake, and adequate checking is at best difficult. 
One scheme of working, in which the coefficients of the various values 
ol Jj in iormula (5.31) are first calculated and checked and are then used 
to form the sum (5.31). has been given by Comrie an important feature 
is the check of the coefficients which is provided. In another type of 
method, suggested by Ait ken. J the result is obtained by a sequence of 
steps each of which is similar to a linear interpolation; this is considered 
m § .3.7 1. Another way of arranging the work is considered in § 5.72. 


5.71. Evaluation of Lagrange’s interpolation formula by a 
sequence of linear cross-means 

Linear interpolation or extrapolation of f(x) from the values f a ,f b , 
of /at r= x n ,x b respectively, gives 




'(.,-) — (•*'■ ~ x )f u J r(x—x n )f b 


(Cl 


X 


(5.32) 


a 


• >. < i nit 

A. * Aii!:u» 


6-Figure Tables, vol. 2 (1949). Introduction, p. xxxi. 
rru.\ Lain. Math. Soc ser. 2, 3 (1932), 56. 
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Aitken calls this quantity the ‘linear cross-mean’ between f a and f b \ 
let it be written f ab (x). Linear interpolation between the values f atb (x) 
and f bc {x), regarded as values of an auxiliary function at x = x a and 
x = x c respectively, gives 


x c x a 


Let this be written f llfiiC (x). It can be verified that this is the value of 
f(x) given by a three-term Lagrange interpolation formula using the 
values of/ at x = x a , x b , andr c . 

In general, let be a set of numbers obtained by successive 

use of the linear cross-mean formula 

f _ ( x k~ x )fa.b.e...i,i( x ) + ( z ~ X a)fh.c...i.j.k( X ) (5.33) 

Ja,bfi...ijA X > - X k -X a 

(n suffixes) 

Then it can be proved by induction that f a ,b.c~'i.j.k( x ) 1S the value of 
f{x) given by an n-point Lagrangian interpolation formula using the 
values of f&tx = x a , x b ,... t x jt x*, which need not be in monotonic sequence. 

The process suggested by Aitken consists of forming the linear cross- 
means f 0i i(x),f 0t 2 (x),f 0 i {x),... and in general f 0 j(x), then using these to form 

foA.i( x )* fo.\,z( x )’— and in general / 0 u (:r), then / 0 ., i2 , 3 (x), fo.t.2.4( x )’—'fo.i.2j( x ) 
and so on. A graphical representation of the formation of the first 
set of linear cross-means f 0J (x) is shown in Fig. 7. An alternative 
order of procedure! consists of forming first / 0i i(*)> f^ix).... and in 
general f u+1 (x), then f 0X2 (x), f 1X3 (x),..., f u+w+ ,(*> and so on. The forma¬ 
tion of the first set of linear cross-means f fJ *i(x) m this procedure 
is shown in Fig. 8. 

In using the latter procedure, the work can be arranged as follows: 
x x — X: f, yL + i.xd-') 


X 

X-Xj 

Si 

Su+iW 

x O 

x-x 0 

' Jo 

So.A*) 

*1 

X - X ll 

/, 


r 2 


ft 

/».3<*> 

*2 

X — X 3 

J 3 

SjJgS' 



- 



/•.».*<■*■> 

I UaaI') 


So. ) 


This layout is similar to that oi a uinercncc w>u.c, ..“ 

not di(Terences. The entries used in forming f w > xi (x) are enclosed in 

t E H Neville. Journ. Indian Math. Sor. 20 (1934). 87. This paper includes an 
extension of the procedure to use known values of derivatives of/(x). 
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‘boxes’, and the way in which they are selected is shown by the lines 
joining them to the entry 




Fig. 8. 


This process has the advantages 

(i) successive calculations are all repetitions of this simple process 
of forming linear cross-means; 

(ii) the results provide their own criterion of when the process has 
been carried far enough; 

(iii) common leading figures in two of the linear cross-means can be 
suppressed in taking higher cross-means. 

The process can be used on functions tabulated at equal intervals of 
the argument, but takes no advantage of this uniform tabulation. 
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5.72. Divided differences 

For the treatment of polynomials at unequal intervals of the argument, 
it is sometimes convenient to use what are called ‘divided differences . 
Ifx Q , x v ..., and in general x Jt are the values of x at which the function is 
tabulated (not necessarily in monotonic sequence), the first-order 

divided differences are 




x,-x 0 


•• M 


f lx x ' /(»t+l)-/(*<) 


(5.34) 


The second-order divided differences are 

/ (• c i> x t) /(^o» *^1) 
#2 *0 


/(*<>, *i, * 2 ) = 


f{Xj, Xj +1> Xj +i ) 


x j+2 x i 


and in general, the nth order divided differences are 

f[X M , X 1+2 ,---> x j+n)—f( X i> X J+ l'" 1 ’ Z J+"-0 

f( x j,x j+ 1 »-»*/+») — x } + n - x ) ... 

(5.o5) 

For a function tabulated at equal intervals (Sx) of x, the divided differ¬ 
ences become the quantities 8"//n! (to)-. The mam P^perty of dmded 
differences for practical purposes is that the nth order divided differences 
of a polynomial of the nth degree are constant. 

Another applet ion of 

fo^ulw Such formula for equalty^J „ hich i9 equivalent to 

interpolation formul derivation does not tako advantage 

Lagrange’s interpolation formula ; but- this derivator ^ ^ doeg ^ 

of the equal spacing of t e ar ^^^ fficient8 in interpolation formulae such ua 
exhibit the relation e tho coe ffi c ient8 in tho expansions considered in 

ft? °Th!"fon Tthese formulae by the use of finite-difference operators 

seems preferable. , 

To show that the nth order differences of a poljmomial rfthajrth 

degree are constant, consider first some properties of'*7^l^ ences 
Tim relations (5.35) defining the successive orders 

are all linear in the function values/,, hence any divided difference is a 
linear combination of a set of function values with coefficients which are 
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functions of the xf* only. It can easily be proved by induction that if 
j < k < j+n , the coefficient of f k in f {x f ,x J+v ...,x i+n ) is 

1 ft' {*k- x i) 

1 i=j 

the dash in FI' indicating that the factor with i = k is omitted from the 
product; if k is outside the range j to j-fn, the coefficient of f k is zero. 
The value of this coefficient is unaltered by a change in the order of the 
factors, and it follows that for any function /(x) the value of a divided 
difference f(x Jt x i+v ... t x J+n ) depends only on the values of x i involved, 
and not on the order in which they are taken. 

Now consider the function / (a;) = x n . The first-order divided differ¬ 
ences are given by 




= x j + l+ x j x j 




n -2 

f+1 


-f- xj 1 ~£j+i 




n —1 


a homogeneous polynomial of degree (n — 1) in x j} x j+1 . Similarly 
f( x jy x j*v x ‘j+z) is a homogeneous polynomial of degree (n — 2); and by 
induction it can *be shown that f (Xj,x j+v ...,Xj: m ) is a polynomial of 
degree n — m. For consider the difference 


./ (•*; ►!»•••» #> jJt» ' ¥ j+tc+ l) f( x j* X j+V"> X j+k)‘ 


(5.36) 


Since divided differences are independent of the order in which the values 
of Xj are taken, this difference is 

f( J j+k+ 1’ x j +!*•••» x j+ x j+k)f 

and this is zero if r ; _ A . rl = Xj. Hence if the Arth order divided difference 

f{ r j- r j+i . r j i.) is a polynomial in Xj,...,x j+k , the difference (5.36) 

contains {Xj, k rl —Xj) as a factor, and the divided difference f (Xj,x }+V ..., 
•'V i) is a polynomial of degree one lower than f (Xj,x j+1 ,...,x j+k ). 

Xmv lor f(x) -- .r". /(.r/,^-j) is a polynomial of degree (n — 1), so 
J x i 2 ) is » polynomial of degree (n-2), and so on. In particular 

1 . r ; n) is a polynomial of degree zero, that is a constant; it 

is therefore independent of the values of x Jt ...,x J+n , and has the same 
value as it these were equally spaced, namely, 1. 

Tor a polynomial of the nth degree with leading term a 0 x n , the nth 

icdei divided differences of all terms but the leading term are zero, so 

t lie nth order divided differences of this polynomial are constant and of 
value n 0 . 

Th" result- that for a polynomial of degree n, the nth order divided 
differences arc constant, can be used to verify whether a set of (n+w) 
va, ""‘ s ol V *' im litt-ort by a polynomial of the nth degree. It can also 
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be used to determine values of this polynomial for other values of s, 
and so to carry out interpolation. 

The latter calculation can be done by a process of building up from 
nth order divided differences, rather in the way in which a polynomial of 
the nth order can be built up, at equal intervals in .r, from its nth differ¬ 
ences (§ 4.42). Further, it is possible to determine derivatives of this 

polynomial, as follows. 

If x J+l = x,+€ t then f(x Jt x i+l ) = /'(*/)+<>(«), and in the limit c -> 0, 
f(x.,x.) = /'(*,). Although/ (Xj,Xj) cannot be evaluated directly from 
the values of / and the definition (5.34) of divided differences, it can be 
built up from higher orders of divided differences and so determined 

in this way. Similarly 

f( x j> x j> x j) = 2\f 

and in general f( x i ,x it ...,x i ) = — f {n) (Xj). 

7i -f 1 urgurnonts 


Example: To show that the following values of/(x) are consistent with/(x) being 
a cubic in x, and to find/(6),/'(6),/"(G) for this cubic: 


X 

/ 


-1 

-11 


o 

l 


2 

1 


3 

1 


7 

141 


10 

501 


The working can be arranged as follows: 

1st order 


x 

- 1 


/ 

-11 


2nd order 


3rd order 



The working above the inclined line is concerned with showing that the third-order 
divided differences are constant, as is necessary for a cubic; that. below the^hne * 
concerned with the evaluation of this cubic and .ts derivatives at « - J. The 
indicate the sequence in which the numbers in the lower part are obtained. 
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In the first part, the divided differences of successively higher orders are calcu¬ 
lated directly from the definition; for example: 


/(- 1 . 0 , 2 ) 


0-12 
2 —(— 1 ) 




In the second part, the divided differences of successively lower orders are built up 
from those of higher orders. The value 6 of a: for which f(x) is wanted is written 
as the next value of x in the table. The value of/(3,7,10,6) must be the constant 
valuo 1 for this cubic, that is 


/(7,10,6)—/(3,7,10) 

6-3 “ 


so that /(7,10,6)-/(3,7,10) = 3. 

This value is added to/(3, 7,10) = 15 to givo/(7,10,6) = 18. Then 

/(10, 6)—/(7,10) /(10, 6)—/(7,10) 


/(7,10, 6) = 


6-7 


-1 


so that /(10, 6)-/(7,10) = - 18 

and this is added to/(7,10) = 140, to give/(10,6) = 122. Finally 


/( 10 , 6 ) 


/(6)—/(10) /(C)—/(10) 

6-10 -4 


so that/(6)— /(10) = —488, which added to/( 10) gives/(6) = 73. 

This value ofy(G) can he checked by taking it, with the values of/(3),/(7), and 
/(10). ns given values of the cubic, and using them to obtain/(2) in a similar way; 
the value obtained should reproduce the value which was used in forming the 
divided difference table used in the evaluation of/(6). 

To obtain/ '(6) we put a second value of 6 for x, and repeat the process as far as 
the first-order divided difference only, and for A/"(G) we put a: = 6 again and repeat 
the process as far as the second-order divided differences. The result gives the 
cubic in powers of (x— 6); in this case 

/ = /(6)-!-/'(C)(ar-G)4-i/’(6)(a : -G)=+i/'"(6)(a:-6)3 
= 73 + 54(ar-6)+13(x-6) 1 + (a:-6) 3 , 

and the evaluation of this for the values of x for which the function values are 
originally given checks the whole calculation. 


5.8. Inverse interpolation 

The problem of inverse interpolation is this: given a table of f(x) as 
a function of x, to find the value of x for which/(.r) has a specified value. 
If the table is not at equal intervals in x, there is no distinction between 
direct and inverse interpolation; the following applies to tables at equal 
intervals in x, as is the case in almost all tables. 

'1 he. table can be regarded as one of x at miequal intervals of/(:r), and 
a method of interpolation of functions given at unequal intervals of 
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the argument (§ 5.7) can be used for inverse interpolation. This process 
takes no advantage of the equal intervals in x t and needs care in use; 
an example of how not to use it is given below (§ 5.81). 

Bessel’s formula to third differences, using modified second differences, 


is /*=/(* o+0$*) = /o+«*/i+w)( 8 »/o+ 8 */i)+ B,1, ^ )8 %; 

in inverse interpolation, f(x 0 +9Sx) is given and this equation is to be 
solved for 9. If the third-difference contribution can be neglected, it 
is a quadratic for 9, and could be solved as such, using the conventional 
formula, but this is a laborious and unsuitable method for practical 
work; it has been said that ‘nobody but a mathematician would do it 

that way’. 

One method is to determine 6 roughly by means of a graph or a lew 
trial direct interpolations, and then make a small subtabulation jn the 
neighbourhood of the rough value of 9, at such an interval that linear 
interpolation can be used for the final step. This method may be found 
the best for occasional isolated inverse interpolations, and in the neigh¬ 


bourhood of turning values of/(a:). 

Another method is to write Bessel’s formula in the form 

9 = [A-/ 0 -W)(5^/ 0 +8» i / 1 )-5* ,, WS 3 /i]/8A < 5 - 37 > 

and use an iterative method, improved if required by the process of 
‘ exponential extrapolation’ (see §§ 3.4 (a) and 9.32). If second differences 
are modified by the use of the throwback from fourth differences, as 
indicated in formula (5.37), this can be used provided fourth differences 
are less than 500. If they are greater than this, it would be best to 
proceed by means of some preliminary subtabulation, for values of 9 

in the neighbourhood of that given by (5.37). 

The accuracy to which 9 can be determined depends on the number 
of figures in 8/* and in assessing this accuracy it must be remembered 
that the last digit of 8/* may be affected by rounding errors to the extent 
of ± 1. Thus a value of 8/* of about 200 is necessary to establish a second 

decimal in 9. .... 


Exam pi 

c: Given 

the following 

table, find sin '0-4 

X 

f(x) = sin x 

8 V 

0 

0 

17365 

0 

-528 

10° 

0 17365 

10837 

- 528 

— 511 

20° 

•34202 

15798 

-1039 

-480 

30° 

•50000 

14279 

— 1519 

-435 

40° 

50 J 

•G4270 

0-70604 

12325 

- 1954 


8V 

o 

17 

31 

45 


0-1848V 
0 

-3 

-G 

— 8 


8m/ 


0 


531 


- 1015 


152 
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For interpolation in the interval x = 20° to 30°, we have 



34202 

15798 

-2572 


S 3 /i = - 480 

in terms of the fifth decimal as unit. We want 0 for/ = 0-4, f—f 0 = 5798 in terms 
of the fifth decimal. Hence, substituting in (5.37), 

0 = [5798 + 2572£«(0)+48O£U'(0)]/15798 
= O-367O 1 + O-1628£»(0) + O-O3O4£»»(0). (5.38) 

A nominal fifth decimal is kept here in (/—/ 0 )/8/j, but not in the other terms since 
the quantities £*‘(0), £ iU (0) by which they are multiplied are less them -&• 

The first term in (5.38) is the value of 0 which would be obtained by linear 
interpolation. Taking it as a first approximation to 0, the iterative process is as 
follows: 

0 r.h.s. of (5.38) 

•367 •3670 1 -f(0-1628)( —0-05808)-f(0-0304)(-f 0-0051) 

= -3670 1 --0094 6 + -0001 5 = -3577 0 
•3577 -3670 1 + (0-1628)(-0-05744) + (0-0304)(+ 0-0054) 

= -3670,— -0093 s + -0001 6 = -3578 2 . 


The change in the value of the right-hand side of (5.38) is only about of the 
change in the value of 0, so the value 0-3578, would not be changed by more than 1 
in the fifth decimal (due to rounding errors) if the right-hand side were evaluated 
for the better approximation 0 = 0-3578,. The number of figures in 8/j is not 
enough to determine the fifth decimal in 0 to several units. According to the 
purpose for which t he value of sin -1 0-4 was wanted, it could be rounded off to four 
decimals, or the fifth retained as a guarding figure; if the latter course is taken it 
would be advisable to write it as a suffix, as a reminder that it is subject to an 
uncertainty of several units. Thus the result would be written sin -1 0-4 = 23-578 2 °. 


F« >r the worker who is fortunate enough to have the use of two machines 
simultaneously a convenient way of carrying out this successive approxi¬ 
mation has been devised by Comrie.t 

fare is necessary when carrying out inverse interpolation near a 
stationary value of the tabulated function. In such cases it is advisable 
to carry out it preliminary subtabulation so that in formula (5.37) 
(8 2 /,+ B-f ; is not greater than \Sf- v before carrying out the interpolation. 


5.81. How not to do inverse interpolation 

The following example illustrates the dangers of trying to carry out 
inveise interpolation by using a Lagrange interpolation formula for 
x in terms of f i.r).J 


See r 1 


amlirss G-Figure Tables, vol. 2 (1949), Introduction, p. xxix. 
t fho wanting provided bj this example seems necessary, as this method has boon 
recommended without qualification in a book on finite differences, and, moreover, in a 

on text very similar to this example. 


4 
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Example: Given the following table 

x = 0 1 2 3 4 

y = 0 1 8 27 64 

find x for y = 20. 

The five-point Lagrangian formula for x in terms of y is 

r(y— 8)(y — 27)(y—64) j ( (y- l)(y-27)(y-64) 

* “ y [ I. ( — 7)( — 2G)( —63) ’ + 8.7.(— 19)( —56) 

. (y-l)(»-g)(y-64) (y-l)(y-8)(»-27) ,1 39) 

+ 27.26.19.(-37) ' T 64.63.66.37 J 

and evaluation of this for y = 20 gives x = - 1-316, instead of the correct value 
(20)* = 2-71442. The result is not appreciably improved if one takes a six-point 
formula by including the value x = 5, y = 125 so that there are three points on 
each side of the value for which the interpolation is carried out; and a better result 
(x = + 2-923) is obtained if a three-point formula involving only the values x = 1, 
2, and 3, is used in preference to the five-point formula. 



The reason for this discrepancy is that x — y* cannot be represented 
adequately by a polynomial in y over the range in question, whereas 
Lagrange’s interpolation formula is based on a polynomial approxima¬ 
tion to the function being interpolated. Fig. 9 shows a comparison 
between x = y* (full curve) and the quartic polynomial (3.39) by which 
the Lagrangian interpolation is carried out (broken curve). 
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If this method is used for doing inverse interpolation, it is advisable 
to check the resulting value of x by doing a direct interpolation in the 
table of f(x) for that value of z, and to verify that this reproduces the 

value of f{x). 

5.9. Truncation errors in interpolation formulae 

Except for polynomials of degree not greater than n, an interpolation 
formula to wth differences, or an (w-f-lj-point Lagrangian interpolation 
formula, is only an approximation. All the formulae for direct interpola¬ 
tion which we have considered express the interpolated value / (x) of a 
function as a linear combination of tabular values / (Xj), or, to put it 
another way, as the result of some linear operation on the function 
specified by these tabular values. A general method for finding a formal 
expression for the truncation error in such a formula has been given by 
W. E. Milne, and is considered in the next chapter (§6.7). 

The method of inverse interpolation considered in §5.8 is not linear 
in/, and is not covered by Milne’s treatment. 
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6.1. Definite and indefinite integrals, and the integration of 
differential equations 

There are two kinds of situation in which we may want to carry out 
numerical integration. One is the integration of a given function of the 
independent variable; this is sometimes called quadrature.-}- The other 
is the integration of a differential equation, which can be regarded as 
the evaluation of an integral in which the integrand at each value .of * 
depends on the value of the integral at that value of x. This is represented 
formally by writing the solution of the equation dyjdx = f{x,y) as 



( 6 . 1 ) 


From the point of view of carrying out the integration by numerical 
(or mechanical) means, the only difference between quadrature and the 
integration of an ordinary differential equation is that in the former 
case the integrand in (6.1) is independent of y , and so is known as a 
function of the variable of integration over the whole range of x before 
the integration is started, whereas in the latter case the integrand at 
any value of x is not known until the integration has been taken as far as 
that value of x. The present chapter is concerned with the integration 
and differentiation of given functions of x. The integration of differential 

equations is considered in Chapter VII. 

In integration of a given function of x, the results required may be of 

two kinds, a definite integral J /(x) dx between a single specified pair 

a _- 

of limits a, 6, or an indefinite integral j /(£) d£ as a function of its upper 

limit The latter is much the more important, and will be considered 
first ’ Usually when results of this kind are wanted, they are wanted at 
the same values of x as those at which/(x) is tabulated, though occasion¬ 
ally results at twice this interval will be adequate. s 


t It is also sometimes called •mechanical quadraturebut tins term is ^loadmg 
since there is nothing more mechanical about the process than there rs about any other 

numerical calculation. 
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6.2. Integration formula in terms of integrand and its differences 

The relation between the first differences of a function and its first 
derivative and the differences of this derivative has already been 
obtained in § 4.74, where it has been pointed out that such a relation is 
an integration formula. We have 

fi fo = ]f'dx; 

*0 

substituting this in (4.58) and replacing/' by/we have 

jfdx = t(Sz) [/„+/,-, l t(S‘f,+S*f 1 )+ 

+ S(8 1 /o+SV 1 )- ! j&(8 6 /o+8 s /i)]+0(3 J: ) 9 . (6.2) 

This could be obtained directly by the use of finite-difference and 
differential operators, without reference to § 4.74, as follows. Expressed 

X.l 

in terms of operators, J / dx is (E— l)Z) _1 /o> and we want to express this 
in the form l(Bx)<f>{8){E+ 1)/ 0 . Hence 


or 


4(&r)<£(8)(£+1) = ( E-\)D ~ 1 

# 8 ) = 1 = tanh ^ 

' E+l U W 


and the algebraical work of expanding <f>{6) in powers of 8 then proceeds 
as in § 4.74. 


An Alternative derivation is by integration of Bessel’s or Everett’s interpolation 
formula with respect to 6, tor 

2*1 i 

I Jdx = (Sx) j f(x 0 +68x) dd. (6.3) 

X 0 0 - 0 

I n ! b .-sel’s formula, t he coefficients of the odd order differences are odd functions of 
(e .A. so they give zero on integration. The integrals of the coefficients of the 
c. ■ ti -order differences in Bessel’s formula give the coefficients in (6.2). 

The ratio of the coefficients of 8*/ in (6.2) to that of 8 2 /is - JJ = -0-1833, which 
i- very close to the value -0-184 used in modifying second differences in inter¬ 
polation by means of the throwback; this is not surprising in view of the close' 
r< hit Km just men* ioned between the interpolation and integration formulae. How- 
’ / unl "^ modified second differences have to bo calculated anyway for intcr- 
purposes, use of them in the integration formula is no simpler than 
- ‘douhtu-ig the fourth difference contribution as it stands. 

hi using the integration formula (6.2) it is advisable to add the con¬ 
tributions in the square bracket -first and finally multiply the whole by 
,:/ie r ‘' l fher than dividing each separate contribution by two 
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before adding; this halves the possible rounding error without requiring 

that any additional figures should be kept. 

For reference later, an alternative form of formula (6.2) should be 
noted. From the relation (4.34) between the operators p and 8 it follows 
that (F+1)S 2 “ = —l)/zS 2 ” -1 . So each pair of terms (S 2 "/ 0 +S "/i> 

can be written 2(pS 2 '- 1 / 1 -p8 2 "- 1 /o) and formula (6.2) can be written 

alternatively 

J fix = (Sx)[i(/ 0 +/,) — + 

*• < 6 - 4 > 

This is not as convenient as (6.2) for integration through a single interval, 
but may be more convenient for integration over a number of intervals. 


6.21. An alternative derivation . 

Formula (6.2) and other integration formulae can be obtained by a 

rather different approach as follows. The simplest , 

often known as the 'trapezium rule’ or the 'trapezoidal formula , 

(6.5) 


| / dx = i(S.c)(/o+/i) 




( 6 . 6 ) 


For a more accurate formula, let us write 

J fdx = £($*)[/o+/i+ C 'd ; 

J(Sx)C, can be regarded as a correction to the result obtained by the 
trapezium rule. 

Now Jfdx = (E-l)D~'fo “ nd (/»+/.) = < £+1)/ “ 

so that C x is given in terms of operators by 

Different integration formulae 1 given by different ways of expressing 

° v a ( < ; n terms of the sum of contributions from the 

be^^SXin" C write it as the result of an operation 

on (£+l)/o> thus: 

.of .h, .1”"” ‘ ei ™ 


(6.7) 


6303 
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It is also possible to expand it in powers of U and so obtain an integration 
formula in terms of higher derivatives of /, but this is of little practical 
value as these high derivatives are seldom available. 

If we want to express Qin terms of the difference between contribu¬ 
tions from the beginning and end of the interval, we write it as a result 
of an operation on (E— 1)/ 0 . One way of doing this is to use in formula 
(6.8) the relation (4.34), namely (£ , -fl)8 = 2(E— l)/x. This gives 

n 2 TtanhH/ .1. 

and expansion of (tanh \U){\U in powers of 8 then gives (6.4). 


6.22. Integration formula in terms of the integrand and the 
differences of its derivative 

Another way of writing C\ as a difference between contributions from 
the beginning and end of the interval is 


c _I 

1 D 


2 E -f 1 
U E -1 


(£-l)Z>/ 0 = ^ 


j](6.9) 


This form for C v in terms of the derivative /' of the integrand and its 
differences, is convenient as the operator here is an even function of 
U and so of 8. 

From formula (4.34), (E+l)/(E—\) = 2 /i/8, so formula (6.9) for <?, 
can be written 

= -(2/8=)(8a-)[(/z8/^)-(8 2 /^ 2 )](/;-/o). (6.10) 

I he expansions for (fiojU) and (8 2 /I/ 2 ) in terms of 8 are given by 
(4.52) and (4.47) respectively; substitution in (6.10) gives 

o’uS 2 4- 0 A 0 S 4 - 3 ol^oS 6 ](/i —/o) -4- 0(8a;)«, 

so that 




J7( 


\r)dx = l(8.v)[(f { B +/ 1 )-S(S^){(/;-/i)-e , o(8 2 / , i-3 2 /;)+ 

-F & i«(S 4 /i—Sy o)—s«^oo(5 6 /;—)}] -h 0(Sar)io. (6 . n) 

An advantage of this formula is the small coefficient (J.^ = 3 J 0 ) of 

the term of order (8.r)» in the square bracket, compared with that { f} 6 ) 

of the corresponding term in (6.2). Further, the term of order (S.r) 2 "in 

the square bracket only involves values of quantities at the beginning 

and end oi the interval of integration, whereas the corresponding term 

in (6.2) involves the values of /, and /_, outside that interval. We shall 

see later that both these advantages are important in the integration 
of differential equations. 
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6.23. Integration formula in terms of integrand and its deriva¬ 
tives 

Expansion of the operator in (6.9) in terms of L instead of in terms 
of 8 will give an integration formula in terms of the integrand and its 
derivatives. This is known as the Euler-Maclaurin formula. 

In terms of U, (6.9) becomes 

C x = — { 2 (Bx)lU 2 }[\U cothW— l](/i— /o)- 

Now the expansion of izcot \z in powers of z isf 


( 6 . 12 ) 




(6.13) 


the coefficients B n being the Bernoulli numbers; the values of the first 
few are 


Bo = 1 


B, = l 


B x = i Bo = a V B 3 = i. 2 , #4 - so. 

The corresponding expansion of ^coth^, is given by putting z = iy 
in (6.13), so the required expansion of (2/L -2 )[H cot i .> ] is 

= < cu) 

Substitution in (6.12) then gives an expression for C', and substitution 
of this in (6.6) gives the Euler-Maclaurin integration formula 

]}(x)dx = l(8x)[/ 0 +/,-l(8x){(/i-/i)-.U^Wi"-/o) + 

(6.15) 

This formula is of limited practical value, since values of the higher 
derivatives of the integrand will not generally be available f bey may 
however, be available in two eases; first, when the integrand » gn en b> 
a sufficiently simple analytical formula, and secondly, when / satisfies 
a sufficiently simple differential equation. The analytical formula oi 
differential Equation must be such that it can be differentiated severs 
times without leading to expressions too complicated for practical 

numerical evaluation. 

6 3 Integration over more than one interval 

If it is adequate to obtain the values of the integral at intervals 
greater than those at which the integrand is given, other integration 
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formulae are available. For integration over 2k successive intervals 
(Sz) we Have 

Xk 

j f(x) dx = = 2(smh kU)D~ 1 f 0 . 

X-k 

A first approximation is 2 k(8x)f Q} so we try to obtain a <f>(S) such that 

J /(*) dx = 2k(&x)<f>(&)f 0 . 

X-k 

The operator <^(S) required is therefore given by 

2k(Zx)4>{8) = 2(sinh.kU)/D 

or <f>(S) = (sinh kU)JkU. (6.16) 

In particular, for k = 1 (integration over two intervals), 

^(8) = (sinh U)/U. 

The expansion of this operator in powers of 8 has already been considered 
in §4.75; substitution from formula (4.52) gives 

■o 

f /(*) dx = 2(Sa-)[/ 0 + J8 2 /o-Tfe8 1 /o+n'n8 9 /o]+0(Sx)». (6.17) 

« * 

J-l 

The first two terms give the finite-difference form of the integration 
formula usually called ‘Simpson’s rule’. This can be seen by expressing 
o-f u in terms of function values; then the first two terms in (6.17) give 

\f(x)dx = 2(3*)[/ 0 + Ufi-Zfo+f-J] = J(8*)[/ x +4. / 0 +/.J, 

1 ^ (6.18) 

the usual form of Simpson’s rule. 

Another important formula of this kind is related to the result of 
putting k — 3 in (6.10). For k = 3. 


<^(8) (sinh 'SU)/3U = l + W*+ilU*-i&U*+0(Sx) 6 

= [l + eS 2 +^ 4 +fi tm+0(8x)*. 

11' now we replace the coefficient fa by fa = & we obtain a formula 
certainly not correct to sixth differences of the integrand but in which 

■ iuKca the whole of the contribution from the sixth difference is included, 

po moiy 



/(•") dx ••= i 3 «(^)[20/ 0 -f 



1 1 5Vo+8 fl /o]-ii 0 (Sx)S«/ o +O(S a :)», 
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or, in terms of function values, taking only the terms in the square 
brackets, 


|'/(x) dx = i 3 e (3^)[/ 3 + 5 /2+/i + 6 /o+/-i + 5 /-2+/-3]- (619) 

I-l 

This is known as ‘Weddle's rule’. 

Another procedure for evaluating an integral over a number of equal 
intervals is to express it as the sum of a number of trapezium-rule 
contributions and a correction. 

The approximation to J fix) dx as the sum of a number of trapezium- 

Xo 

rule contributions over intervals hx is 


Wx)(f 0 +2fi+2h+- + 2 f"-i+f'') 

= £(8x)(l -t-i?)(l E" M/o 

= l(8x)[(E+l)(E n -\)l(E- l ))fo- 

The integral itself is (JS»— 1 )/>-»/.: Iet us write il as th ® “ um ° f th ° 
trapezium-rule contributions, plus a correction H&x)C n \ that is, 

(£”-l)Z>-‘/o = j /W dx = i( S:c )[§^l (£ ' ,_1)/o + C, ‘]' 


Then 


0. - (§ -< *— 1 >A - - K) 


so that C n is related to (/,-/„) in just the same way as C x is to <£-/•> 
(see § 6.21). Thus we can write down three integration formulae directl> 

from the results of §§ 6.2 to 6.23. 

In terms of the integrand / and its differences: 


7 fix) dx = ( 8 x)[J(/„+ 2 /i + 2 / t + ... + 2/„_ 1 +/„ ) - 1 l i (F-S/,-^ ) + 

In terms of the integrand, its first derivative, and the differences of this 
first derivative: 


j f(x) dx = A(8a-)[/ 0 + 2 /i + —-\--fu-i+fn 

-i(&){(r.-r.)-A( s v'.- s v'.>+«w s v'-- 84/i) --" > ]- (,i - 21) 
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% 

In terms of the integrand and its derivatives (Euler-Maclaurin formula): 

Xn 

/ A *)** = 1 (&c)[/ 0 + 2/1+2/2+•••+ 2 / n _ 1 +/ n — 

— i(&r){Cr M —/i)—&(«*)*(/?—/T)——}]. (6.22) 

The last is of limited practical use as a formula for numerical integration 
for reasons already mentioned in § 6.23. It is, however, useful for 
numerical work in another context (see § 11.12). 

6.4. Evaluation of an integral as a function of its upper limit 

The evaluation of an integral as a function of the upper limit can be 
carried out by successive addition of the contributions from a sequence 
of intervals (Bx) covering the relevant range of x. Let us for brevity 

X 

write F[x) for j /(£) d£. Then if, for example, the integration formula 

a 

(6.2) is used, these contributions are 


hF i+\ = J f(*)dx = h(8x)s J+i , 


(6.23) 


where s >+J = ; (6.24) 


X« 


and j /(.<) (/.vis the sum of « such contributions. The summation of these 

X 0 

contributions can be expressed by use of the central-sum operator a, 

the inverse of 8 (see § 4.6); operating on both sides of (6.23) with a we 
have 

F J~ F 0 = MM[(<W) ; .-(<7S) 0 ]. (6.25) 

It values of the integrand are available at half the interval Bx at. which 

t he values of the integral are required, a convenient alternative formula 
for s i+i , based on (6.17), is 

•'■V* = /;• 1 , * - iir,5 \f t + , + j - • • •, (6.26) 

uliere b indicates the central-difference operator for intervals \Bx. 

In summing coot ributionsof the form (6.23), there will bean accumula- 

1,0,1 of ro,,r . ,din « om,rs fmm ,ht ‘ corrections to the trapezium rule for the 
successive intervals. This accumulation can be made unimportant by 
the use of a guarding figure in the contributions (6.23). 

however the calculation of the integral is done, it must be checked. 

, ,U ‘ ,a,ls ° f tlu ' checking procedure depend on the method used to 
evaluate the integral; the following procedure, given as an example, 
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refers to a calculation carried out by evaluating the successive con¬ 
tributions (6.23) and adding them. It is then advisable to carry out one 
check on the values of s j+l and another on the evaluation of the integral 
from them; the intermediate check of the values of s, ., will avoid the 
possibility of a large number of values of the integral having to be 

corrected if one of the values of s j+i is in error. 

Operating on both sides of (6.24) with S 2 we have 

ga s . + , = (8 2 /„+S 2 /.) - + S ‘/> > + 5=« (8 “- f »+ S<! /i) - 

A comparison of the values of S’a calculated (i) from successive values of 
s by the method of §4.43. and (ii) from formula (0.27). provdes a good 
check on a values, and a clear indication of the location of mistakes^ 
any. When this check has been made and mistakes, if any. cor 
and only then, the values of should be summed, and the 
integral F calculated from (6.25); the multiplication by ,(»x) should 
follow the summation, otherwise it may be necessary to keep an 

figure to avoid rounding errors. , , f .. 

This summation and multiplication by J(M can be checked as follows. 
From (6.23) we have 8 3jp = .1 (5x) 8 2 s j+i , (,i 2S) 

and values of V are already available as they have been used in the 
process of checking the values of a. The values of the m eg a Fean be 
checked by comparing the values of PF obtained 0 as « S'* horn he 
values of F, the second differences being evaluated by the method of 

§4.45 and then differenced once more, and (n) rom on 

Various alternative checking procedures of this kind can be let e,l. 

for example by taking the fourth differences of both s!des o ( -4 
give a check of the values of .. Another kind of check ,s an mcr 
check by one of the methods considered in § 6.5; but this is - ; 

since although it will indicate the presence of a mistake it will not usilalh 

locate it. 

Example: To evaluate f dz to five decimals at intervals 0 1 in -r. 

The first seven inters of the calculation are e™! - "n- 

column, rather than each row, refers to a Mnp c \ a “ . diiTt.To l icM.- s 

calculated from formulae (0.24), thecontri m ions ° |j iu > below 

just appreciable; the second differences of ^^Z of formula (11.27) 

the vulues of s themselves, and the hums o ^ , G | \ between those 

occur four lines lower down. T he discrepancH -• * mistake, probably 

two sets of values at * = 035. 0-45. and 0-55 clearly indicate amnUaikj f 

of ,10 units in the las, figure, at X = ^read - 5K3,. ... 

8 1 /, 1 8 l J Ux ) for the interval 0-4 to 0 o, "nun 

«(0-45) = 2-45180. 
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The differences 8 2 a affected by this correction should be recalculated and the 
comparison with formula (6.27) verified to make sure that the correction itself has 
been rightly made. In actual working the corrected values could be written in 
place of the erroneous values, but in this example they have been written separately 
to display both the incorrect and correct values. The successive values of a are then 
summed, and the sums multiplied by \(8x) and rounded off. Finally, this summa¬ 
tion and multiplication is checked by comparing 8(8 2 F) with £(&r)8*«. 

Notes: (i) In the check comparisons between the values of 8 2 s calculated by the 
two ways, and in the similar comparisons for 8 3 F, discrepancies of a unit can be 
expected as the result of accumulated rounding errors, but discrepancies in succes¬ 
sive values should usually be in opposite directions. 

(ii) The contribution of S 8 / to 8 2 s may be appreciable, although its contribution 
to s itself is negligible, and similarly for other orders of differences. 

(iii) In this case / is an even function of x, hence each difference of even order 
is an even function of x, so the even-order differences at x *= 0 can be calculated 

from the formula g2"+2/(0) = 2[8 2n /(8x)-8 2n /(0)]. 

To obtain the second and higher differences at x = 0-8, the values of f(x) up to 
/(l-l) have been used, though to save space they are not given explicitly. 

(iv) Each value of / is subject to a rounding error of up to \ in the last figure, 
so each value of s is subject to a rounding error of up to 1. If these rounding errors 
were randomly distributed, the ‘probable error’ (in the technical sense of tho term 
us used in the theory of errors) of the sum of N values of a would be approximately 

so that of the result would be about JA'*(8.r) in the last significant figuro of the 
values of the integrand. In the present case, with N = 8, this is rather less than 1 
in the sixth decimal in the integral. In making estimates of this kind, it must be 
remembered that errors up to 2 or 3 times the ‘probable error’ are not unlikely. 

(v) Another method of calculating this integral, more convenient for large values 
of x. is given in the example in § 7.3. 

(vi) The checks indicated verify the calculation from the values of the integrand 
/ but they do not check these values themselves. The differences of the values of 
t he integrand form a check against gross errors, but do not check the last digit with 
certainty. An overall check on the whole calculation, including the values of the 
integrand, i« provided by doing an integration by one of the methods of § 6.5 using 
another set of values of the integrand, say in this case by integrating from x = 0 
to 0-8 using ten steps of length 0 08, or by using a Gauss integration formula (see 
5 6.57). 

6.41. Change of interval length in an integration 

It may sometimes be required to change the interval length (Sz) in 
the course of an integration. Where the third and higher derivatives 
of the integrand are large, it is advisable to take small intervals, not 
only in order that the corrections to the trapezium rule should not be 
too large, but in order that the behaviour of the integrand should be 
adequately defined by those of its values which are used in the integra¬ 
tion formula. It may happen that the values of the third and higher 
deris atives vary considerably over the range of integration. If in' such 
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a case the interval length (Bx) which is necessary when these derivatives 
are large were used over the whole range, this would involve an un¬ 
necessary amount of work in the region in which they are small. So a 
change of interval length, or several such changes, may be required in 
the course of the integration. 

When such a change is made, there should always be an overlap 
between the ranges of .t for which the different sizes of interval are used. 
This is a potent check against mistakes, which are particularly likely 
to be made at points like this at which a systematic procedure is inter¬ 
rupted. It provides a check not only against random mistakes but against 
some forms of systematic mistakes as well; if, for example, the term 
-i*(S 2 / ; +8 2 / ;+ i) in (6.24) had been taken systematically with the wrong 
sign, this would be shown up by a discrepancy between the integration 
carried out with two different interval lengths. 

In practice, the convenient intervals are 1, 2, and 5 times a power 
(positive or negative) of 10; an increase of interval length from 2.10« 
to 5 . 10 f ' involves some interpolation, but this is all of the simplest 
‘halfway’ interpolation (see §5.21) and should give no trouble. 


6.42. Integration in the neighbourhood of a singularity of the 

integrand 

A point at which / or any of its derivatives becomes infinite will be 
called a ‘singularity’ of/. 

In deriving the integration formulae of the previous sections, it has 
been assumed that the integrand f{x) is expansible in a Taylor series 
through each interval Sr. This is not the case if there is a singularity at 
any point (including end-points) of the interval, and the approximations 
used are likely to be bad in the neighbourhood of a singularity even if it 
does not lie in the interval through which the integration is being taken. 
Examples are: 

f c ~ nx 

.1 sen 

1 

(ii) | r* sin .t dx (second derivative of integrand infinite at x = 0). 

0 

A singularity can often be removed by a change of independent 
variable; for example, the substitution x = cosh u makes 

I ————- dx = I e~ acoshu dit, 

J (*“-!)* J 

l a 


dx (integrand infinite at x = 1); 
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and usually such a change of independent variable will also give results 
in a more satisfactory form for interpolation But if the results, . 

required in terms of*, and in a context in which no '" tcr P^ t '° n ‘ l ‘ 
carried out on them, it may sometimes be better to obtain them directlj 

" a” singularity can sometimes be removed by the following process 

We subtract from the integrand / a function S which can 

formally and has a singularity of the same kind as /, er .luate he 

analytical formula for j g dx, and evaluate /(/. 0) * - 

integration. This may be called ‘subtracting ou the smgtW . 

This is satisfactory if the singularity is a pole of order „, but * 
it may not be possible to remove the singularity m h,s way, though 
may he made less severe. For example, we can write 

-—nr / 


c p~ ni „ f dx 

j “ c J i ^ 2 - 1 


-)i+ 


f dx. 

J (v 2 -D‘ 


The integrand in the integral on the left is infinite 
that in the second integral on the right is finite, mug 

'"ai^ alternative, and often more effective, ^ ^ 

Write the integral \ }(x) dx = **>»<*>. " h ^ » IS This leads 

which dg/dx has a singularity of the same nmn( . rica | integra- 

r;“^‘^lo. ) Tlt.s may be called ‘dividing out the 

singularity’. 

Consider, for example, integrals of the form 

|\r"/(.r)r/.r, 

_° , ri(\\ o Jin( l « is greater than — 1 

where/(at) is regular at * _ 0 and/ - ■ '■ f func tion <j{x) is 

and is not an integer. For this case the appropn 

x i>+*Hji+ 1), so we write 

J *»/(*) 

0 

and on differentiation obtain 

x' 1 - = (;>+!)(/-*)■ 
dx 

On differentiating k times and putting x -= °> * ,1 ' s " IVCS 

A7t (W (0) = (p+Dt/^Ol-W)], 

pl- 1 y<*>(0). 


((*,. 20 ) 


(0.30) 


so that 


h tk \ 0) = 


(0.31) 


p -f-1 +k 
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In particular 


These serve as starting values for a numerical integration of equation 
(6.30). 

Numerical integration of equation (6.30) is also useful as a means of 
evaluating integrals of the form J x p f(x)dx for values of the lower limit 
in the neighbourhood of zero when p is negative and \p\ > 1. 


m = m 

a '(°) - 

A'(0) = «±i/'<0) 

p + 6 


(6.32) 


6.43. Integration when the integrand increases ‘exponentially’ 

A similar device of writing an integral in the form 

J f(x) dx = g{x)h(x), 

choosing a convenient function g(x), and solving numerically the result¬ 
ing differential equation for h(x) can often be applied when the integrand 
increases rapidly with x, particularly when log/(a:) increases more 
rapidly than linearly in x. This process then consists of dividing out the 
singularity at infinity. One good choice of g(x) is the leading term in 
the asympotic behaviour of/ (x); this makes A -> 1 asx -> oo. For example, 

| e jt dx behaves asymptotically like e xi /2x, so we may write 

6 

[ e x 'dx = {e xt /2 x)h(x) (6.33) 

6 


then 




(6.34) 


This, however, is clearly not convenient for small x, and in order to 
obtain an equation applicable to the whole range of x we may be content 
to divide out the main part of the singularity at infinity by writing 




(6.35) 


a simpler equation than (6.34), and one which there is no difficulty in 
integrating numerically from .r = 0 (see the example in §7.3). 


6.44. t wofold integration 

By a many fold integration of a function / (*) is meant a result obtained 
by repeated integration with respect to the same independent variable, 
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as distinct from a double integral which is obtained by one integration 
with respect to each of two independent variables. That is, a twofold 
integral of a function / is a function F such that 


d?F 

dx* 



It is sometimes convenient to be able to obtain such a twofold integral 
directly without going through the intermediate stage of evaluating \fdx. 
This can be done by using formula (4.54), which for this case becomes 

8= (Sxfl/o+^/o-^Vo+s^/ol+O^)’ 0 - < 6 ' 36 > 

The twofold summation of the second differences S *F to give the func¬ 
tion values F can be done either directly by the method of § 4.4b or 
by obtaining the first differences as an intermediate step and then 

summing these. . 

Effects of rounding errors may build up somewhat rapidly in this 

twofold summation, and it is advisable to carry some guarding figures. 

The twofold integration could be carried out by two single integrations, 

one from F" to F' and the other from F' to F. Suppose that in an 

integration carried out by this method, n decimals would have been 

kept in F'. Then in a twofold integration carried out by a method not 

involving the calculation of F', the number of decimals kept in 8 F (or 

in F if 8F is not calculated) should be enough to give n decimals in 

8F/8*. 


6.5. Integrals between fixed limits 

An integral between fixed limits can be evaluated by any of the 
formulae of §6.3 or §6.4, the difference being that the value of the integral 
is only wanted for a single value of the upper limit. 


0-8 


Example : To evaluate j e z * dx using formula (6.21). 

In this case/'(x) = 2xe>*' is an odd function of x, hence all even -order ‘fdTcrences 
of /'(x) at x = 0 are zero, and their contribution to formula (6.21) is zero, 
even-order differences at x = 0-8 we have the following values: 


x /'(x) = 2xe 7 
0-5 1-2840 

•0 1-7200 

•7 2-2852 

•8 3-0344 

•9 4-0462 

1-0 5-4366 

M 7-3776 


8*/' 

8 1 /' 

8 •/' 

1292 

1840 

238 


2026 

374 

50 

3786 

5506 

560 



At x = 0-8 

/' = 3 0344 
= - -* 3 . 

+xi a 8 r = + 

-Sum =“3-0301 
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The values of f(x) are given in the example in § 6.4 (p. 105); using them we have 

2 2/0‘Sx) = 17-33644 
i~ l 

/(0)+/(0-8) = 2-89648 

and, from the values of/' above, 

= —05050 
20-18242 

Hence jV dx = i(0-1)20-18242 = 1-00912,. 

0 

The tolerance on this result, due to the accumulation of rounding errors of the 
function values used, is a few units in the sixth decimal. 


Some other forms which are only appropriate to an integral between 
fixed limits are considered in the following sections. In particular, there 
is the possibility of using values of the integrand f{x) not spaced at 
equal intervals in x if there is any advantage in doing so (§6.57). If 
the integrand is specified by a table at equal intervals in x, then an 
integration formula which makes use of this feature is usually the more 
convenient; the interpolation necessary to give its values for use in an 
integration formula using unequally spaced values of x would usually 
outweigh the advantages of such a formula. But if it is specified by a 
formula which can equally well be evaluated for any value of x, then 
use of values of x at unequal intervals may become practicable. 


6.51. Gregory’s formula 


The integration formula (6.20) expresses the correction C n to the sum of a set 
of trapezium rule contributions in terms of central differences at the beginning 
and end of t he-range of integration. If the only available values of the integrand 
an- those from /„ to the ends of the range of integration, then only forward 
differences from the beginning and backward differences from the end of the range 


are uv 


ailuble, and a formula in terms of these differences is needed. This is 


f/(.r) d.c 

■To 


(8x)[ *(/(,+ -/l + 2/»+ - . + -fn-l *f 
-r- ( h A - + A P o A 3 - ... )/ 0 - 


/n) + 

(AV + *Y* + VAV3+ ...)/„) (6.37)' 


ami is known as Urtijury'a formula. 

Its practical use is limited, because usually the reason for the limitation on the 
range over which a function is defined is the occurrence of a singularity, as at 


x 4- l in J c~ l2 ( 1 — .r 2 )~* dx 9 iu which case the extension of the integration 

formula up to those points is invalid. In most other cases, values ot the integrand 
outside the rang*' of integration, and therefore the central differences required in 
formula (6.20). are available, and then this formula should always be used in 
preference to Gregory’s formula. 


INTEGRATION (QUADRATURE) AND DIFFERENTIATION 

6.52. Integral in terms of functi °" or (6 . 37) m terms of func 

By expressing the differences in ; * t | lc integral as a sum of mult iples 

x-“ jjsm. 

For example, if in (6.20) we substitute . 

/x8/ 0 = i(/i-/-i). A = /n - l, 

and neglect terms beyond these, we obtain 

)' m dx = ,ta, I -*/-,+*/.+HA+/.+™+/«+«^* +i/ *-* / " , i 8) 

*• tv,, e differing in the order of differences 

Coefficients in a number offormulae o ‘ function values outside the range 

to which they are correct and in the nt TMea j 

*. to *„ used, are given in Cl ‘°’" b " s J ’ in tcr n,s 0 f differences, such as (6.17) 

The writer’s own preference is foi form into account, and 

as it is much easier to see which differences have ^ 

inclusion of an extra one does not mean altering 

6.53. Use of Simpson’s or Weddle * bl . taUen into account by using 

TheSVterms in the integration fom.u^( n ) lliatl , a<1 of by using the term 

Simpson’s rule for the inter\a s i tai'e this procedure, except the 

Wn-tff.) in (6.20). There is no ^ ^disadvantages. It is 
smaller coefficient of the 8*/ term, an. trapezoi< i tt l formula for each pair of 

equivalent to calculating a correction o w j, cn on ly a single value of an 

intervals in the integration, whic 1 is u, * n , - t Evolves a substantial amount of 
integral between fixed limits is requm < . identically, as. for example, in 

work in calculating a correction winch may tan,. ■ 

« . alternate funct ion values, and r« quirt s 

f e~ x * dx. It also gives different weights to alu 

o ver the range of x shoul 1 bo even, 

that the total number of intervals require* io coy m fact that the number 

Use of Weddle’s rule has similar 
of intervals has to be a multiple of 6 ma\ 

CO 

6.54. The integral J e“** dx 

° t the limits of U(x)dx, then 

If nil odd derivatives of /(-» ) ure zcro a ° u . 

,* of integration to the correction 

the total contribution from that end of t u "* n ^ l “ |p Q(1(1 derivatives are zero at 

to the trapezium rule formula i> ro - ‘ '* Fuler-Macluurin formula that 

both ends of the range, it would appear Iron , f , aZoi( ial contributions 

expression for the integral » the sum of a num ^ ^ . 

is exact, whatever interval length (o) 

. • . if r - z 'dx Since the integrand/(x) is a 

An example is provided by the integral j 

t Vol. 2, |». 54‘J. 
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even function of x, every odd derivative is zero at x = 0; also all derivatives are 
zero at x = oo. Hence the Euler-Maclaurin formula appears to give 

7 e - * a dx = (8x)h 4 ( 6 - 39 > 

o L j-l J 

exactly, for any value of (8x). If both sides of (6.39) are evaluated for different 
values of (8x), the results are as follows: 


CO 


j e~ xt dx = Jtt* = 0-88622 69254 5 to eleven decimals 

e\ 

■1 


(8x) 


(8x)[ 


i + 2 - 

i-l J 

0-5 0-88622 69254 5 to eleven decimals 

0-6 69254 8 

0-7 69285 

0-8 0-88622 72808 

0-9 23 598 

1-0 32 0 

1 1 0-88674 

Thus the relation (6.39) is nearly true for values of 8x which are quite considerable. 
Why it is not exact is that the Euler-Maclaurin formula is only asymptotic. We 
saw in §4.6 that the finite difference operators, and Taylor’s series in the form 
E c L \ could be used freely on functions which were the products of polynomials 
find exponent ials of linear functions of x; e - * 5 is not of this form, and an examination 
of the *-rror t<-rin is necessary before the Euler-Maclaurin formula is applied to it 
with (8x) so large that the approximation to the integrand in each interval by a 
sum of products of polynomials and exponentials becomes dubious. Such an 
examination has been carried out by Goodwin.t 

The point might seem of formal rather than practical interest, since anyone with 
experience of numerical work, faced with the values of e ~ T * at intervals of, say, 
1, namely: 


c 

OX 


X 

/ = c~ zt 

8/ 

8 2 / 

8*/ 


0 

1 

-6321 

- 12642 

— 15467 

-30934 

1 

0-3679 

- 3496 

4- 2825 

+ 489 

415956 

2 

0-0183 

— 182 

3314 

- 3133 

- 3622 

3 

oouoi 

- 1 

181 

- 180 

4 2953 

4 

0-0000 


1 


179 


5 


0-0000 


would say that these function valuta alone did not define the integrand well enough 
to just ify e\ a I rat ing the integral to more than two significant figures at most; it is 
hardlv necessurv actually to form the difference table to reach this conclusion. Of 

v % 

all the functions which take tluv>e tabular values, there is presumably at least ono 
for which the Eisit-r- Madam in formula happens to give the right value of the 
integral; t hat • • 2 so nearly has this property seems to a certain extent accidental. 

t E. T. Goodwin, 1'roc. Camb. Phil. Soc. 45 (1949), 241. 
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However, the fact that it has this property raises .he 
of relatively large intervals (Sr) in the evaluat.on of other integrals 

J .-../(*) dr may also give results of useful accuracy. Tins also has been examined 
0 

by Goodwin.t u 

6.55. Evaluation of a definite integral when the integran 

singularity w hich the integrand has a 

In evaluating an integral J/(•*)«■* 1 indenen- 

singularity, the singularity can often be removed by ^ t , 

dent variable. If the integral is required as «t ft"f^^^Xut 
limit, we may want to avoid this in order to " “ ' values of 

further interpolation, values of the in egi« * limits and in 

x. But this does not apply to an integral betwcen ^ a 

this context the only reason for avoiding a c wi c obtain the 

certain amount of interpolation may ire required m 

values of the integrand at equai ^ ^^variable over 

There is, of course, no need to use the new mac per 

the whole range. For example, to evaluate j 1 ^) d - 

use the substitution over the whoie range of , and so evaluate 

the integral as i \ 

y /* /*/ I - . _ o* J.. 


C fix) dx = 2 f m-y*)dy 

J (l -*) 1 J 


or we might divide the range of x into t \n > I sU l,stitution in the 

and the other from * - * to 1 and only make t »■ ; - 

second part, thus evaluating the integia in ^ ^ 

r /<*> dx _ ijMix+i J &-?)*»■ 

J (1=5? j ( 1 - I) » , or 

0 0 ,i,. t i_f is a perfect squaie. 1 

In this case £ should be chosen so 1 enable intervals of 0-04 

example £ = 0-64 might be taken, t ib 1 j s ( ,jo 1 in y in the 

or 0 08 in x to be used in the first integral and mtuv. 

second. arameter 

6.56. Definite integrals which are funC j IO " 1 S ^ ni ^. ol npris«'S those 
An important class of integrals bet-we< n ^ the integrand, 

which define a function of a parameter which occuu* 

b 


such 


f c ~ y,< - <}■» or generally <j( x ) | - ,( 

J (I—**)* i 

t E. T. Goodwin, Proc. Carnb. 1‘lnl. Sot. 4$ ( 


(0.40) 


MM 
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Such an integral can be evaluated by quadrature for each value of x, 
and this may be the only way of evaluating it. But another method of 
treatment may be much easier and less laborious if it is possible at all. 
This consists of finding a differential equation which the integral (6.40) 
satisfies, and solving this differential equation by a numerical process 
(see Chapter VII). The amount of work required to obtain a single 
value of the integral is then very much less than that required to carry 
out the evaluation by quadrature, and probably evaluation by quadra¬ 
ture will only be carried out for two or three values of x, to give initial 
values for the integration and to provide an overall check. It is not 
always possible to obtain such a differential equation, but many integrals 
of this kind of which the values are actually wanted in various contexts 
do satisfy differential equations. In such cases, the differential equations 
can often be obtained by one or two differentiations with respect to x 
and an integration by parts with respect to u. 

Consider, for example, the function 

CO 

J(x) = J [c-“*/(i< + x)]du 
0 

which has been studied by CJoodwin and Staton.f The range of integration hero is 
infinite, but for x > 0 the integrand is of such a form that differentiation with 
respect to x is justified. One differentiation gives 

co 

/'(*) = - JV“V(«+x) 2 rfu 
0 

ami integration by parts with respect to u then gives 

co 

/'(.r)= | c" ul ^[l/(u + x)]r/u 
0 

co 

= [c-“ 5 (U + Z)]®_ 0 + J2M[C-“V(M + X)]rfu 

u 

[ <? CO 

J e~ ui du — a* J {e““*/(“+#)} du I 

--= — (1 /.r) -f — 2xf(x), 

so that this function/(.r) satisfies the differential equation 

f' + 2xf = 7Tl-(\/x). (6.41) 

Evaluation of/(.r) by quadrature for one value of x % say x = 1, could be used to 
give u value from which the numerical integration of equation (0.41) could bo 
started, though Goodwin and Staton actually used a series expansion to obtain 
such a value. 


1 Quart. J. Mech. ami Applied Math. 1 ( 1948 ), 319 . 
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6.57. Use of unequal intervals of the independent variables 

As already mentioned, in evaluating integrals between fixed limits 
there is no need to use values of the integrand at equal intervals m z 
and there may be advantages in using formulae in terms of some ot her 
set of values. An integration formula using a finite number of values 
of the integrand can be regarded as giving a weighted mean of these 

ValUe8 ' |/(*)ie = (6-o)[Zw»/ ( *»)]- (B - 42) 

a 

where the <s are the weights to be assigned to the values of the inte¬ 
grand at the points x„. Given any m values of not necessarily equally 
spaced values of ». can be obtained which will make such a formula 
correct for any polynomial of degree up to (m-1). And it is possible to 
put a condition on the w.'s (such as that they should all be equalsand 
determine the corresponding *„’• such that this formula should be 
exact for polynomials of degree up to (—D- But . no condition ,s 
imposed on either the a„'s or the <s, then these can be determined so 
that a formula (0.42) with m terms in the sum will be exact for any 
polynomial of degree (2m-1). Such a formula is known as an m-po.nt 

Gaussian integration formula. # r . .1 

For work on numerical evaluation of integrals of given functions the 

practical value of such a formula is limited by two tilings; first, the 

necessity for interpolating the values of the integrand at the required 

values of and. secondly, the difficulty of checking both these values, 

whk , are unequal intervals in *. and the evaluation of the integral. 

Further no formula which is based on a polynomial approximation 

can be appropriate to an infinite range of integration. 

On the other hand, use of such a formula may be very valuable n 

simplifying problems in more than one variable. It can be used, fo 

example, to simplify integro-differential equations involving mtcg.aU 

of the type } J(r,B)smO JO. If such an integral is replaced by a sum 

I u.(8;)/(r, 0)),°the solution of the equation in which the integral occurs 

can be reduced to the solution of a finite number of equations for the 
functions /(r,8>), each of which is a function of the single lo 

only In making such a replacement it is clearly des, able to obtain a. 
g :ld an approximation as possible with a small number o erms a 
this is given by taking the values of 0, and the weights ««,) to be those 

of a Gauss formula. 
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The values of £j = a)/(6—a) 

and of the weights w s for values of m from 1 to 5 can be found in Whittaker 
and Robinson’s Calculus of Observations .f 
For m = 5 they arej 

^=1-^ = 0046910 w x = w s = 0-118464 

Zz = 1—= 0-230766 u' 2 = w t = 0-239314 

£ 3 = 0-5 tt> 3 = 0-284444 

° a 

Example : To evaluate J e + x * dx by a five-point Gauss formula. The values of x s , 

o 

the integrand values, and weights are as follows: 


• 

J 

x i 

Si = expx? 

Wj 

1 

•037528 

1 001409 

•118464 

2 

•184612 

1 034669 

•239314 

3 

•400000 

1173511 

•284444 

4 

•615388 

1-460388 

•239314 

5 

•7G2472 

1-788475 

•118464 


Tn'jfj = 1-261401 (6 — a) = 0-8 

(b-a) V wjfj = 1 009121 

Hole: This value of the integral agrees with those calculated in §§ 6.4 and 6.6. 

b.t>. Numerical differentiation 

We have already seen that a table of values does not define a function 
uniquely. Still less does it establish whether the function tabulated is 
differentiable everywhere, or even anywhere, within the range of the 
table; two functions may be indistinguishable, to any specified degree 
of numerical accuracy, for every value of x (not only for the tabular 
values), yet one may be differentiable everywhere and the other nowhere. 
And still less does a table establish whether a function is differentiable 
two or more times. These considerations alone suggest that numerical 
differential ion of a function specified by a table may be a dubious process. 

Further, the entries in a table are affected in an irregular way by 
rounding errors, and differentiation exaggerates irregularities whereas 
integration smooths them. In differentiation we are concerned with a 
limit process carried out on the quantity [ f(x-\-Bx) — f{x)]/(bx), and as 
o.r becomes smaller, irregularities in the values of / become more pro¬ 
minent in the result; whereas in integration we are concerned with a 
limit process carried out on the quantity 2/S.r and the effect of an 
irregularity in any one value of / becomes less prominent as Sx becomes 


+ See § hO. 

I Tins values uvo roumleJ oil so as to have the sum 1*000,000. 
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smaller. This is illustrated by Fig. 10, which shows the graph of the 

function lsinlOrT* _ 100x * 

V = 2 1 0-nx 6 



which might represent approximately an irregularity P^iml 
rounding error at < = 0 in a table at intervals of 0-1 m *. and the 

tivc and integral of this function. i i. v 

For such reasons, the different iation of a function ^oificd ’ > 

a table of values, or determined experimentally and so subject to erro, 
^observation, is a notoriously unsatisfactory process 
higher derivatives than the first are required. It » a process 
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avoided if possible, unless the context in which the results are required 
is such that the limited accuracy attainable by the numerical process 
is certainly adequate. In general, values of the second and higher 
derivatives obtained from such data should be regarded with caution if 
not scepticism. 

In some cases it may be possible to evaluate derivatives by some 
process other than numerical differentiation. If, for example, a function 
y is known to satisfy a first-order differential equation, values of y' can 
be obtained by substituting the values of y into the differential equation. 
And if y satisfies a differential equation of higher order, it is usually 
better to obtain y' by numerical solution of the equation as an equation 
for y' , taking y as a given function of x, than to try to obtain y' directly 
from the values of y by a differentiation formula. 

6.61. Differentiation formulae 

To obtain a first-order derivative there are two useful formulae, one 
giving the values of the derivative /' at the values Xj of x at which the 
function is tabulated, and the other giving/' at x j+i . The former has 
already been obtained in §4.71, and is (see formula (4.46)) 

(M/i - MS/o- l^o+^ 5 f 0 -l\o^ 7 fo^O(8xr 
= M(/i-/m)-S(S7i-87-i)+^ 

(6.43) 

The other can be obtained either bv differentiating Bessel’s interpola¬ 
tion formula with respect to 0 and then putting 8 = I, or by using finite- 
difference operators as follows. We want to find a (f)(8) such that 

(S-r)/; = *(8)8/*. 

Hence 

f f, { 8) r= CIS = (sinh-U5)/»8 = l-£8H»?t£ 4 --»ftiS«+0(8z)« 
on putting n = 1 in (4.42). Hence 

(S-r)/i = 8/j - £8% + eJ 0 $ 5 /» - 7 WV) + 0(8x)\ (6.44) 

This formula is much preferable to (6.43) on account of the more rapid 
decrease of the coefficients of the higher orders of differences. If, 
however, values of f) are required, there is no advantage in using 
(6-44) followed by half-way' interpolation between the values of f' j+i 
by use of formula (”*.3). since these two processes together just give 
formula (0.43). 

For a second-order derivative, the appropriate formula is (4.43), 
n! " ,U ‘ ly <»*>*/; - «=/»-y/o+ i , , o5 6 /o- i .l< i S 8 /o+0(S^)‘». (6.45) 
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In carrying out the calculations, the interval (fa) taken should not be 
too small since the smaller it is taken, the smaller the number of signifi¬ 
cant figures in 8/, and so in/' (and similarly for a second demat nek 
Rather Sr should be taken as large as is convenient, subject to the 
truncation error of the differentiation formula used being negligible. 

It wm often be advisable either to smooth the values of / before 
differentiation or to smooth the values of/' or/' obtained (for a smooth- 
fng process see § 11.4). Let us write/; for the smoothed values which form 
. .*• t f T 0 ensure that no systematic errors are intro 

a due a ed P inTe a smoothing process, the values of/; should be integrated, 
and compared with the original values off. If the quantities/- j f,dx. 
Which are called the ‘residuals’, show any significant systematic % anati 
whht aprocess of differentiation should be carried out on these residuals. 

—# - “ ", 

withr(0)= TzW == _ 1 l^to find !/'(0) and i/(2) for the solution = 1 + ryfor 

which 2 /( 0 ) = 0,2/ (0) = 2- 


x 

10 

1-7 

1-8 

1- 9 
20 
21 

2 - 2 
2-3 
2-4 


Y(x) 

1-56205 

1- 83254 

2- 14428 
2-50475 

2- 92297 

3- 40984 

3- 97855 

4- 64506 

5- 42876 


8 -Y 8 *Y 5 6 Y 


4125 

4873 

5775 

6865 

8184 

9780 

11719 


154 

188 

229 

277 

343 


7 

7 

18 


z(x) 


2- 80444 

3- 17749 

3- 61107 

4- 11708 
4-70978 


2 '( 2 - 0 ) 

= 4-07626 


_ , .■ _ ri i linvo been tabulated.t The} 

-rLJL « cl this example , refitted to 

them by 2 (x) = [Bi(x)- 3*Ai(ar)]/2.3»/5, 

where fi i« a constant given in the Ihaf Introduction), 

of 2.31(3 is 0.890577, this function-(*>-‘‘X^cakulutcJ from .hi. formula; 

«•>»* ho rcsulls fiOU6ht - but ne,8,,bounn8 
ij = Y+CZ, 

where c is an arbitrary constant, for the solution with /(M) = "• 

C 7= -r(2-0)/z'(2-0), 

„ . , - f , rubles Part-volume B <1940). 77. e /!<>!/ 

f British Association Mathematical I 
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so wo need to determine *"(2 0). From the tabulated results and formula (6.43) 

0-2F'(2-0) = 0-90509 —i(2409)4-^o(89)—T*^(H) 

= 0-90509-40IS + 3O-0! = 0-90110 4 , 
while 0-2z'(2-0) = 0-935252, so c = -(0-90110 4 )/(0-935252) = -0-96348*. 

With this value of c we have 

x Y -cz y 

1-8 2-14428 — 2-70203 = —0-55775 

1- 9 2-50475—3-06145 = —0-55670 — 58 

2- 0 2-92297 —3-47920 = —0-55623 —113 —I 

2-1 3-40984-3-96673 = -0-55689 -169 

2-2 3-97855-4-53779 = —0-55924 

and for this function y,/(2-l)-/(l-0) = - 00019 and 8*/(2-l)-5*/(l*9) = -1H, bo 

0-2f/'(2-0) = —00019— i( — 111)-F — 1) 

= - 00019+18 6 = --00000 6 , 

which is within the tolerance for rounding errors. This provides a check of the 
work. 

Notes: (i) The value of i/(20) is not determined correctly to a unit in the last 
figure; the value c = -0-96347 5 gives 0-2y'(2-0) = +-000 00 5 . which is equally 
within the tolerance for rounding errors,>tmd 2/(2 0) = —0-55621. 

(ii) The value of i/'(0) for this solution is just »/'(0) = c. 

(iii) The value of I"(2-0), is not determined with certainty to several units in the 
fifth decimal, since r/(2-1) — ?/(1-9) is subject to rounding errors up to 1 in the fifth 
decimal and is multiplied by l/2(8x) = 5. A more accurate vuluc of F'(2-0) 
could bo obtained by using 8x = 0-2, but the contribution to 2(8x)F' from the 
higher orders of differences would be considerably greater; that from 8 6 F, for 
example, would be greater by a factor of over 100. 

6.62. Graphical differentiation 

The residuals /— I f' s dx of a numerical differentiation over a range 
of c will usually be numbers of one or two digits only, so can easily be 
plotted to the accuracy to which they are known. In such a case, a 
graphical method of carrying out the differentiation is adequate. The 
best way of doing this is to plot on good squared paper (see §2.5) the 
values of the function to be differentiated, and through each plotted 
point draw a vertical line to indicate the range of uncertainty, due to 
rounding error or other causes, of that value. Then draw the smoothest 
curve passing each plotted point within the indicated tolerance. The 
'latitude in drawing such a curve will give an indication of the reliability 
of the values of the derivative. 

The best way of finding the gradient of such a curve, or of one represent¬ 
ing a set of results of some experiment or observations, is as follows. 
Take a flat piece of polished sheet metal (aluminium or stainless steel 
is satisfactory), or surface-aluminized glass, mounted in such a way 
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that it can be placed on a piece of paper with its surface accurately 
perpendicular to the paper and extending right 
down to the paper. Set this so as to intersect 
the curve at the point at which the gradient is 
wanted (see Fig. 11), and rotate it until there is 
no discontinuity in direction between the curve 
and its reflection in the minor. With care, this 
setting can be made with considerable accuracy, 
probably greater than that to which the cur\c 
can be drawn. The gradient of the curve can 
then be determined directly from the inter¬ 
sections of the plane of the mirror with the grid 
lines of the paper in which the curve is plotted. 

6.7. Errors of interpolation and integration formulae 

W. E. Milnef has given a convenient general method for obtaining 
formally exact expressions for the truncation errors of formulae for 
interpolation, integration, etc. 

We have derived and expressed such formulae as results of linear opera¬ 
tions on the function to be interpolated, integrated, or differentiated. 
If we take one of these formulae to a finite number of terms, then the 
remainder after n terms can also be expressed as the result of a linear 
operation on this function. For example, if we take formula (0.11) as 
far as the terms in/', the remainder, which is the truncation error of the 

formula in this form, is 



| / dx— i(8^)[(/o+/i)~ $(&*0(/i — /o)] 

= (8r)[/(£-l)-l(£+l)4-,',(£-1)1' /„■ («•«) 

which is the result of a linear operation R on/. Milne calls an operator 
R ‘ 0 f degree «’ when Rx"' - 0 for m < «, Jte" +l f «. and writes 
R n for an operator of degree n. The purpose is to obtain an expression 
for R n / for any function/. It is assumed that R„ does not involve higher 

powers of U than U"~ l . 

R may contain some shift operators E so that Rf 0 may depend on 
values of/or its derivatives for values of a: other than x 0 . Let be the 
least and the greatest of these; and let a be a value of * less than 
x m . Also let us write <f>„(z) for the function 

<f>„(z) = z n for 2 >0, <M 2 ) = ° for 2 < °. (°- 47 ) 

+ Numerical Calculus (Univ. of Princeton Press, 1949), §§ 90, 31. 
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One form of Taylor’s series to ?i + l terms with a remainder is 

/(*) = fM+(x-a)/'(a) +1 {x-a)Y(a)+... + 


X 

+ ^i (*-o)"/ w (o)+i J/<" +1 >(f)(z-f)» (if; (6.48) 

a 

this form can be obtained by repeated integration by parts, using 
m J /^(fXs-f)"- 1 df = (x-a) m fl**\a)+ f rff. 


The last term in (6.48) can be written 


a 


00 


^j> +i w„(*- odf, 


(6.49) 


since the integrand here is zero for £ > x. 

I he first »-f-1 terms in (6.48) form a polynomial of degree n, so they 
are annihilated by the operator R„. Also since R„ operates on functions 
m so far as they are functions of a-, it only operates on the function <k 
m the integral in the form (6.49): this is the reason for expressing the 
integral in this form, in which the limits are independent of x. Hence 


cr 


Jl - f ( x) = pj/‘ n * n (t) R n *Jz-() df. 


a 


Since a has been chosen to be smaller than the smallest argument .r 
occurring in RJ, it follows that the arguments of all the terms‘in 

nrC IK,si,iNe for *<«-> but (f> n (z) = - for positive 2 , so 

n<t>M $) 0 for f < a . So the lower limit of the integral can be 

replaced by -co, and finally 


cn 


1{ «fW = f / <,,+l, (f)G(f) df, 


— JO 


where 


( ^) = -R ll 6„(x-^ 


(6.50) 


(6.51) 


If *»/(*)« a function of x, then (?(f) is a function of .r as well as off; 

1 f (x) is not a function of .r, as is the case for the operator in (6.46) 
then (7(f) is not a function of x. 

The function (7(f) consists of polynomial segments between the 
values of x, involved in R„f(x), and is zero outside the range of these 
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values. It is also independent of the function on which R n operates, 

\RJ(x)\ < A r .max|/ ln+,, (*)| in x m ^ x < x M , 


hence 


co 


where 


K = J |G(f)|df; 


— CO 


K is independent of the function / on which R operates. 

In many cases G(£) is of constant sign over the range where it is not 
zero, and then a better formula for the error can be obtained. The mean 
value theorem, applied to (6.50), then gives 


cc 


= /"' +1 W j G{() d(, 

— oo 


(6.52) 


where x <X < the integrand of (6.50) is zero outside these limits 

so x must'lie in th(s range. Also for/(*) = /*♦"(*> = 1 

everywhere, so that in this case (6.52) gi\es 


f G(£)d£ = R n x*+ l l[n+l) 


— co 


and hence in general 

R n f(x) = f {n+1) (X)R„x"+ 1 l(n+ 1)!. (6.53) 

As Milne points out'the evaluation of *„»■♦> on the right-hand side of 
(6.53) is often much easier than the determination of the polynomia 
segments of G(£) and their integration. In (6.53), B„x^ can be replaced 
by JRJx-b)"*' for any constant b if this is more convenient for the 

evaluation of this quantity. 

Example: To obtoin a formula for the error of trapezium rule integration 

| / (lx = J(8r)f/ 0 +/il- 


Here i?/(x) = j/dx - J(&r)(/ 0 +/i>. 

which is identically zero for/(x) but not for/(x) = A Hence 7? is an /?,. and 

RMx-Z) --- , (4> l {x-S)dx-lihx)[4> l {x 0 -t) + <l> 

iT 0 > 

For { < is positive over the whole range x - x 0 to x„ so faix-f) = * " $ 

for all relevant und 

1! G',(£) = + 

which is zero, as it should be. For x 0 < £ - = 0 for ^ ^ 80 

110,(0 = 
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For£ > x.^^x — g) is zero over the whole rangeofa-;so G t (£) = 0. Hence altogether 

= i( x i-Z)(*o-£) for x o < £ < x i 

= 0 otherwise. 

Hence for a general function f(x), 

*■/ = if/'(()(*,- 
*0 

In this case G(£) is zero or negative, and is of so simple a form that it is easy to 

CO 

evaluate J G(£) d£ directly. Substitution of £ = x 0 + (j: 1 —r 0 )rj gives 

— CO 

J OfodS = = -i(8x)? J Tj(l-Tj) drj = —&(Sx)\ 

— ® z' 0 o 

Alternatively, taking/(x) = (j — x 0 ) 2 J2\ and using (6.53) we have 

*i(*-*o)72! = ifi(x,-Xo)*-i(&t)(a-? + »5)] 

= A(Sx)[2(*?+x 1 x 0 + x2)-3(iJ+^)] 

= - i7(Sx) 3 , 

so that R+f = —■hf(X)(hx) i , where .r 0 < A' < a-,. 


6.71. Use of formulae for the error 

If a formula for interpolation, integration, etc., is such that the 
operator K of the previous section is of degree n, the error of the formula 
involves the {n ■ 1 )th derivative of the function to which it is applied. 
But we have seen that for a function specified only by a table of values, 
the numerical determination of derivatives beyond the first or second 
is an unreliable process and one to be avoided if possible. Even when 
the function to which the interpolation or integration formula is to be 
applied is given by a formula which can be differentiated, the formulae 
f •[ the higher derivatives may be too complicated to be convenient for 
numerical evaluation. Thus a formula which depends on the values of 
derivatives beyond the first or second is of limited practical use. 



VII 

INTEGRATION OF ORDINARY DIFFERENTIAL 

EQUATIONS 


7.1. Step-by-step methods 

One class of methods for the numerical integration of ordinary differen¬ 
tial equations consists of those in which the solution is evaluated step 
by step through a series of equal intervals in the independent \ ariable, 
so that when the solution has been carried to x = x jt the next step con¬ 
sists of evaluating the change in the solution through the inter\al hx 
from Xj to z J+v In such a process we follow out in the course of the 
numerical work the development of the solution as the independent 
variable increases. For simple equations this can be made a straight¬ 
forward and easy process to carry out; it can be provided with adequate 
current checks to assure the worker that the integration is proceeding 
correctly, and in the writer’s experience it is one of the most satisfying 
forms of numerical work to carry out. 


7.11. One-point and two-point boundary conditions 

From the point of view of a step-by-step process, the nature of the 
boundary and other conditions to be satisfied is more important than 
the nature of the equation itself; and as regards the boundary conditions 
what matters is not what they are but where they are. if all the conditions 
which the solution must satisfy are boundary conditions given at one 
point of the range of integration (usually one end of it ), the solution can 
be started from there with all relevant quantities known; and, apart 
from the possible occurrence of singularities or of instability in the process 
of integration, evaluation of a solution usually gives no difficult y. Such 
conditions are known as ‘one-point boundary conditions, and a 
problem in which the conditions are of this type has been called by 
ltichard8ont a ‘marching problem’ as the solution is obtained by 
marching step by step from the initial data. 

But if some conditions arc specified at. one point, x = «, of the range 
and others at another, x = b (usually x — <ij> will be the ends of the 
range) or if there is a relation between the behaviour of the solution at 
the two ends of the range such as a condit ion that the solution ij should 
be periodic, which for a first-order equation is y(h) = y(a), or some 

t L. V. Kichardnun. Phil. Tams. Hoy. Hoc. 226 (1027). :iUU. 
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b 

integral condition on the solution as a whole such as j y 2 dx = 1 , the 

a 

evaluation of the solution may not be so straightforward. Conditions 
which are specified at two points of the range are called ‘two-point’ 
boundary conditions; a set of conditions at more than two points is 
possible but unusual. A problem in which the conditions on the solution 
are not one-point boundary conditions Richardson has called a ‘jury 
problem’. 

If the equation, and the conditions which the solution must satisfy, 
are linear, it may be possible to evaluate the solution as the sum of a 
particular integral satisfying the conditions at one point of the range, and 
a complementary function. But in many cases such a procedure is a 
formal possibility only and not a useful one for practical numerical 
work; and if the equation is non-linear it is not available. 

A step-by-step solution has to start from some point of the range with 
definite numerical values of sufficient quantities to define a solution; 
for an nth-order equation these will usually be y and its first (n—1) 
derivatives, but they may be the values y 0 , y v y 2 ,..., y n _ 1 at the beginnings 
of the first n intervals. With one-point boundary conditions this point 
is naturally taken as the point from which to start the integration. With 
other conditions it is best to start from the point at which the values of 
the greatest number of values of y or its derivatives are specified by the 
given conditions on the solution. The other starting conditions have 
to be estimated and adjusted, either by trial or by the use of a complemen¬ 
tary function when this is practicable, until a solution satisfying the other 
conditions is obtained. 

We will consider first the step-by-step evaluation of a solution from 
given initial conditions, and later return to the consideration of the 
determination of solutions satisfying other conditions. 

7.2. Second-order equation with first derivative absent 

T he simplest numerical process is that for a second-order equation 
with the first derivative absent 

y" =/(*.*/)> (7.1) 

in which f(x, y) need not be linear in y. This is integrated by using the 
formula for twofold integration from y" to y, without an intermediate 
calculation of y' (§6.44): 

2fi-2y 0 +y-i = S2 */o = + (7.2) 

One procedure will be explained first in some detail; there are several 
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variants of it, some of which will be mentioned later. It will be supposed 
that the term in (7.2) is negligible, so that the integration formula 

is being used in the form 

5 ! l/o = (S*)=[l/;+,«]. (7-3) 

Suppose the integration has reached x = x Q , and we are concerned 
with the integration through the interval 8x to x = x v At this stage 
we have y 0 and ijl = f{x 0 ,y 0 ) and the backward differences from these. 
The procedure is then as follows. Estimate 8 2 y" 0 , and obtain an approxima¬ 
tion to 8 hy Q from (7.3). Add this to 8y_* to give an approximation to 
8y t , and add this to y Q to give an approximation to y v From this calculate 
y\ and hence 8 2 y" 0 = yl—2y” 0 +y-i- Let € be the difference between this 
value of hhyl and that estimated. A change of the estimate of h 2 y" Q by c 
makes a change j , 5 (8x) 2 e in y v If this is less than i in the last figure 
retained in y , the estimate is adequate; if not, the estimate is revised and 
the calculation of the interval repeated; but the interval length (8x) 
should be taken so that this is seldom necessary. A convenient arrange¬ 
ment of the work is as follows: 


8V y 



The quantities above the heavy lines are those which are known when 
the integration has reached x = x 0 ; the quantity 8 2 j/ 0 enclosed in a 
‘box’ is that which is estimated and if necessary adjusted, and the 
arrows show the sequence in which the various quantities are calcu¬ 
lated. 

To start the integration, two values of y are required, and it is advisable 
to have three to provide a check and to give an indication of the values 
of 8V- These initial values will often be obtainable from a solution 
in series without requiring the evaluation of a large number of terms. 
In some cases it may be necessary to carry out a few steps of an integra¬ 
tion at a small interval before starting the main integration. 


Example : y” = (I —x 2 )y, t/(0) = 0, j/'(0) — 1, — 0-1. 

evaluated from the series solution: 

y = X - 1 - \x 3 — -£fX l + 0(J7 7 )• 


1/(0-1) and 7/(02) 
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1 — X' 


sy 


y 


Vy 


(check) 


00 

01 

0-2 

0-3 

0-4 


0-5 


0-6 



Here the numbers above the broken line are obtained from the series solution. 
In this example, y is an odd function of x, so that we have S“X/"(0) = 0, as well as 
the value 5 2 3/"(0-l) = -0 0051. The value 8 2 «/(0 1) = 0 000988 is obtained in two 
ways, namely (a) from the first three values of y” and the integration formula (7.2), 
and (6) from the first three values of y. Agreement between these values forms a 
check on the starting conditions, and also checks that the 8 2 */' term in the integra¬ 
tion formula has been taken with the right sign; this term has the coefficient H-jV 
horo, whereas in the formula for a single integration, the first term in the correction 
to the trapezoidal formula has the coefficient This makes it rather easy to 

make a mistake of sign at this point, and it is as well to have a check that the 
right sign has been taken. 

If the integration has been taken to x = 0-5, the numbers above the full line 
will have been calculated; we will consider the integration through the next step, 
x G o to 0 t>. From the run of the third differences S 3 t/', the next value may be 
expected tv> be about — 71», giving S^^O-o) about —302; a twelfth of this is —25 

v,l ich gives 52 y (o- 5 ) - (0-01 )(0-3896— 0-0025) = 0-003871. 

This is entered m the 8*// column and checked by forming the value of 8 : [8 2 t/(0*4)]. 
By operating on both sides of (7.3) with S 2 , we have 

S-(S *y) 0 = (&r)*[5 2 y; + ASV;]. (7.4) 

and use of this formula provides a good chock on the values of 8 2 ?/; it should be 
noted that the contribution from 8h/£ may have to bo included in (7.4) although 
it is negligible in (7.2). 

The value of //(O f)) is then built up from tlie value of S 2 y{ 0-5), and from it tho 
value of y~(0*t>) is obtained from the differential equation. This value is 

//‘ T ((Mi) = 0-4049, 

v. lienee o'// (0 f>) — — 297; the estimate was adequate and no recalculation of this 
interval n< oosxury. The ^alculat ion has readied the stage from which we started, 
only one interval further on. and a similar calculation for the next interval can 
u^w be under'taken. 

The differences «»i the values of y" form a check against random mis- 
l ikes in these values, and use of formula (7.4) provides a current check 
of the values ol o~y. The other process which needs checking is the 
t wofold summation of these differences to give the solution y. 
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There are various ways of carrying out this check. If values of y have 
been built up by two successive summations, of hhj to hy and of by to y, 
as illustrated in this example, a good check is provided by evaluating 
the second differences of y by the method of §4.45, winch does not 
involve the calculation of first differences, and verifying that the values 
so obtained reproduce the values of 8*y. This check can be applied as 
each value of y is obtained, but is best carried out occasionally, say 
every ten intervals, in such a way as to verify the values obtained since 

the previous check. . f 

Another check, which can only be carried out on a senes of values of 

y, is provided by taking a set of alternate values of y and differencing 

them to second differences, taking the corresponding values of y and 

differencing as far as necessary for use in formula (7.2), and veil }ing 

that these values of 8V, and of y" and its differences, do satisfy formula 

(7.2) with (8x) equal to twice the integration interval. It will usually be 

necessary to use higher orders of differences in y" in this check than in 

the integration, but central differences of higher order than the secon 

are available at this stage. 

Example: To chock the solution of y" = (l-**)y obtained above. Copying the 
values of y” and y at intervals 8* = 0-2, and differencing them, we have the second 

to sixth columns in the following table: 

§v 

-70 


X 

y" 

8V 

00 

0 

0 

•2 

•1933 

-420 

•4 

•3446 

-910 

•6 

•4049 



y 

o 

20132 0 
41023 3 


8 2 y 
759, 


($*)*(y' + A$V-rfT>$V) 

4(1933 — 35 0 — 0 3 ) = 759, 


The last column gives the calculation of S‘jrfO-2). for 8, = 0-2, from tl.o values of 
y- and its differences. Agreement to a unit in the sixth decimal with the value 
of 8b/ is not to be expected; but this figure is only a guarding figuie. 

The interval hx used, and the number of figures kept in the different 
parts of the calculation, will depend on the equation, the data occurring 
in it, and the accuracy required in the results. This example is representa¬ 
tive of the accuracy which it is convenient to keep in many calculations 
concerned with integration of equations occurring in scientific or 
technical problems. The last figure in y in this example is a guarding 
figure only, and could well be omitted if the function /(*,;/) in the 
equation involved some experimentally determined function which is 
not known to better than 1 part in 1000. In working to this accuracy, 
the first estimate of 8 hf in each interval, its division by 12, the addition 
of the result to y", can be done mentally; so can the multiplication by 

MM K 
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(8x) 2 when 8.r is a power of 10, as will often be the case. Then the first 
number written down is (y"+ \.}8 2 y”) or 8 2 y, and the value of S 2 ?/ is 
immediately checked by differencing. Such an integration can be 
carried out quite quickly. 

The smnHer the interval (8x) taken, the better the estimate of 8 2 y" 
can be made, and the smaller the quantity (8x) 2 by which this is multi¬ 
plied. But it is not advisable to take very small intervals, first, because 
the amount of work required to cover a given range of x increases as 
the length of integration interval used decreases, and, secondly, because 
effects of rounding errors in the values of 8 2 y may accumulate rather 
rapidly in the double summation to give y. If a large number of 
small intervals are taken, additional guarding figures may have to be 
taken to ensure that the cumulative effects of rounding errors are 
negligible to the accuracy required in the final results, and this makes 
the amount of work involved increase rather more than proportionately 
to the number of intervals. The interval length (Sz) should therefore be 
taken, roughly speaking, about as large as is compatible with ease in the 
practical numerical working of the integration. As a rough working 
rule it should be taken so that for only about one interval in five does 
the calculation for an interval have to be repeated. If many have to be 
repeated the interval should be halved. 

If 8 2 y" is not too large, then a good approximation to 8 2 y 0 is (8.r) 2 ?/o, 
so that to this approximation, for a function satisfying equation (7.1), 

Ui = -.yo-y-i+(S-r) 2 /(a-o,y 0 )- 

Thus for such a function we always have a good approximation to the 
funct ion one interval ahead of where we know y ". It is this feature which 
makes the numerical integration of such an equation such a straight¬ 
forward process. 

The procedure for two simultaneous equations 

'/"+/(•>-,//,=) = 0, z"+ 9 {x,y,z) = 0 

or more, with all first derivatives absent, is similar. 

7.21. Change of the interval of integration 

It will not always be advisable to keep the same interval length 
throughout an integration. It may happen that the suitable interval 
length (8.r) varies by a factor 10 or even 100 over the range of x to be 
covered: then the use, over the whole range, of the small interval neces¬ 
sary over part of it might make the calculation so long as to be almost 
impracticable. 
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As already emphasized in §0.41, it is advisable at any change of 
interval length to take an overlap between the integrations carried out 
with the two dilTerent interval lengths. Ihc most usual changes of 
interval length are by factors 2, 2\, or To increase the interval length 
by a factor 2, all that is necessary is to take alternate values of y and the 
corresponding values of y", for two or three intervals before the point 
at which the change of interval length is to be made, difference them, 
and check the formula (7.2) for the function values and differences at 
this new interval, and continue as if these were intervals of the integra¬ 
tion at the new interval. It may be necessary to keep an extra decimal 

in y" to get the same accuracy in y. 

To decrease the interval length by a factor 2, some interpolation is 
required, but only of the simplest kind, namely the half-way' inter¬ 
polation considered in §5.21. Suppose that integration has been 
carried out with intervals Sx = h up to x — A, and it is requited to 
continue with intervals hx = U. The integration with intervals Sx = h 
should be carried for one or two intervals beyond .r =X , to give the 
central differences needed in the half-way interpolation. Then y(X-lh) 
and y"(X-\h) should be interpolated and y"(X — \h) also calculated 
from the value of y(X- M) and the differential equation, to check. Then 

8 3 y(X-\h) = ij(X-h)-2y{X-\h)+y(X) 

should be calculated and compared with the value obtained from 
formula (7.2) at the smaller interval. This checks the interpolation of 
y(X-\h)\ this check is most important since the whole subsequent 
integration would be vitiated by a mistake in this value. The integration 
then proceeds, starting from the values of y(X — \h) and y(X). 

Example: To continue the integration of y" = (l-x-)y from x = 0-5, using 
intervals (8x) — 0 05. 

The value of 80/(0 5) for (Sr) = 01 will he about -21L. so the interpolated 
vulue of t/(0-45) is 

y(0-45) — 0-41023,+ 1(10926,) — £( 342 ? + 387,) + *Js( —22, —29,) 

= 0-41023, + 54(33! -45 6J -0 C1 = 0-40440,. 

The interpolated value of //"(0-45) is 

tf( 0-45) = 0-3446+1(450)—220—297) 
r— 0-3446 + 225 + 33 = 0-3704 

(the fourth-difference contribution is negligibh-). whereas the value calculated 
from tin.* differential r«|iiation is 

t/*(0-45) = ll-(0-45)-J(0-4U440,) -= 0-37036 
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which, to four decimals, agrees with the interpolated value. Thus, starting from 
x = 0-4, we have the following values 


X 

y" 


SV y'+iVSV 

y 


S 2 y 

0-4 

•3446 

258 


•41023 3 

5416, 


•45 

•3704 

192 

-66 -3698 5 

•46440 s 

5509 3 

92 t 

■5 

•3896 



•51949 s 




The value of S 2 »/{0-45) derived from the values of y " is 92 16 , the difference between 
this value and the value 92 4 derived from the values of y is within the tolerance 
for the effect of the rounding error in the interpolated value. The integration can 
therefore be continued from these values. 

Note : In the integration of this equation to this accuracy, it would not actually 
be necessary, or advisable, to decrease the interval of integration at this point; this 
case is only considered here as an example of the procedure. 

The treatment of a change of interval length by a factor 2i is similar. 
Suppose, for example, the integration has been taken to x = 0-30 by 
intervals of 0 02 and it is desired to change to intervals of 0-05. The 
values of y and y" at x = 0-25 are obtained by half-way interpolation 
between the values at x = 0-24 and 0 - 26 , y"( 0 - 25 ) is checked as above, 
and y( 0 - 25 ) checked by verifying formula ( 7 . 2 ) at the larger interval. 


7.22. Variants of the method 

There are several variants of this method, some of which can be 
combined. 

Instead of evaluating y" for each value of x, forming I 1 2 S 2 2 / ,r ) 
and multiplying this by {8x) 2 , we could evaluate (8z)-y" and form 8 2 y as 

8 2 i/o = (&f)*y;+A8*[(8* )V]. 

At a change of interval length, the entries in the column of (Sx) 2 ^/” 
would be different for the two lengths of interval. 

Another variant is as follows. If we operate on both sides of formula 
(7.2) with the repeated central sum operator a 2 = 8~ 2 , it becomes 

y n = (8x) 2 [o- 2 ?/£ -f { 2 i/q — aJgS^o]. (7.5) 

If this formula is used, the aggregate contributions from the ijS 2 ^— 2 J 6 8 4 yo 
terms in (7.2) are evaluated separately for each interval, instead of 
being built up from contributions from successive intervals. This avoids 
accumulation ol rounding and truncation errors in these contributions. 

I he process of starting the integration is rather more complicated, 
as initial values for the double sum a 2 y n have to be evaluated. The 
process does not avoid estimation, since if the integration has been 
carried to .r .«‘ 0 , then for the end ol the step from .r u to .r, we have 
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neither y x nor y\ is known at this stage, only the value of ohj\ and the 
relation y\ = f(x v y l ) between y x and y \; so this is an implicit equation 
for y v and unless it happens to be linear the solution of it may well be 
more trouble than the integration process. 

If the equation to be solved is linear, say 

y = f{x)y-\-g(x) ( 7 -°) 

then the solution, to the accuracy given by neglecting the §V term in 
(7.2), can be obtained without any estimation as follows. In (7.2) 

Wo = y'l-Z'So+y-i 


and if y" is given by (7.6) this is 

h% = (Ay^foyo+f-iy-i^i-^o-x) 

so that (7.2) can be written 

[i_^(8x)Vi]y 1 -2[i- 1 y8x)V 0 ]y 0 +[ 1 “iM^) 2 /-iJy-i 

= (8a- ) 2 [/ 0 »/„+J/o+ 2 (7o] > 

or 8 *[{l-A( 8 a;)*/}y]o = ( 8 *)*[/o 00+00+Vo]- ( 7 - 7 ) 

This treatment is usually ascribed to Numcrov;t it has also been given 
independently by Manning and MillmanJ and, as applied to a pair of 

simultaneous equations, by Wilkes.§ . 

Written as a relation between three successive values of y, (<•') i* 

[i-A(8*)Vi]yi = [2+i(8x)Vo]y 0 -[ 1 -iV^)7-i]y-i+(^) 2 bo+i , u 82 i/o]- 

(7.8) 


A method, due to Olver, for correcting for the truncation errors m this 

formula will be considered later (§ 7.52). 

Both these variants require special treatment at a point at " Inch the 

interval length (8z) changes. 


7.3. First-order differential equations 

For a first-order equation the following method, when applicable, 
seems the most convenient. It is based on the use of tin inti_,iuti 

formula = ^x)[(y 0 -\-y\)-l^x){Syl-^%}\, (7>!,) 

expressing an integral in terms of the integrand and the differences of 
i to derivative (see § 6.22). It is applicable if the function / in the equation 

y' = /(*. y) 

t B. Numorov, Full, de VOUtrv. 2 ,,933) ’ 188 

% M. F. Manning and J. Millmun. Phya. Rev. 53 (l.MS). 6,3. 

§ M. V. WilkcK, rroc. Carnb. Phil. Soe. 36 (1940), >04. 
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is cither given by an analytical formula, so that there is no difficulty 


in evaluating 


d l i 

dx 3y (lx dx J c'y 


to any accuracy required, or if of fix and cfj&y can be determined by 
numerical differentiation to the accuracy required in using the formulae 
of the method. 

Two advantages of formula (7.9) have already been pointed out in 
§ 6.22, namely, the small coefficient of the h*y" term in the square bracket, 
and the fact that if this term (and higher terms) are neglected, so that 
the formula becomes 


— IftxWo+y'i )—$(8*)(yI—yl)]. ( 7 - 10 ) 

it does not involve the values of any quantities at points outside the 
interval through which the integration is being taken. 

Further, the values of y" calculated for use in the integration formula 
can be used to give approximations to the successive values of y by use 

of the formula 8»ft - (&DW+4*^] 

with the term in h 2 y" 0 either omitted or estimated. 

If the numbers occurring in the integration are arranged with those 
referring to the same value of x in a column , instead of in a row, the 
work can be arranged in the following example. 


Example: 

y’ = 1 — 2.r y; y = 0 at x = 0 (see § G.43, equation (6.35)). 

In this case y” = — 2xy'—2y. It would be possible in this case to use a series 
to start the integration, but this will not be used, so as to show the procedure when 
the use of a series is not convenient. 


r 


00 

01 

00 01 

0-2 

0-3 

0-4 

- 2*./ 


0 

- 020 

0 -0199 

—0779 - 

-•1696 

-•2880 

0 

J 

1 - 2 ry 

1 

•980 

1 0000 0-9801 

0-9221 

0 8304 

0-7120 

- 2ry 

0 

0 

-106 

— I960 

- -36SS 

-•4982 

- -5696 

-D 


0 

- -200 

-•1987 

- -3S94 - 

- -5652 

-•7199 

Sum 

/• 

= y 

0 

- 396 

0 -395 

— •758 — 

1-063 

- 1-289 



— 

396 

- 395 

363 -305 

- 226 





32 

58 

V-- 

• 

v/ -* y 

• 

i 


1 9Su 

1 9801 1-9022 1 752 

*U 1 

5424 

— y~) 

- i 

!Ml7 

t -III KjCi 

60 51 

38 

Sum 

.S’ 


1 9s7 

1 -9s67 1-9IIS2 1-757 

til 1- 

5462 

hr 

= S(2y) 


I9s7 

I9S67 • 1Jtos2 • 17.1 


15462 

2.7 


u 

•I9S7 

O • 19867 

•38949 

•ut)52a| *71987 


1 2(8 x) 2 (.v’ + 

&syj 


- 790 

- 1506 

-2112 

H-t. ^ 

N-xt 8(2 y) 



•19077 

•17576 

•15464 


Next 2 y 

0 20 


38944 

•56525 

•71989 
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Since in this equation the values of 2y occur in the formula fur //' au<l y", it i> 
convenient in this case to accumulate values of 2y rather than y. 

For the first interval the values of y 0 , y' 0> and y" are available and a first c-t mot" 
of the value of 2i/(0-1) is obtained by using these values in the first three t.-rms of 
a Taylor series; this gives 2y(0-1) = 0-20. From this, approximate values of y'(0-1) 
and y"(0- 1) are found, and integration carried through the interval x = U to "1. 
giving a better value of 2i/(0-1), namely 0-1987, from which better values of y' and 
y" at the end of the first interval and a better final value of y are obtained. I -e 
of this better value of y does not change the values of y and y to the accuracy to 
which they are being used in the integration, and the revised integration thiough 
the first interval can then be taken as the first step in the main integration. 

Suppose now that the integration has reached x = 0-3, so that the quantities 
to the left of the heavy line are known. The run of the second differences of b-y" 
suggests that the value at 0-3 will be about 80, so that 

8 2 [2y(0-3)] = (0-01)2[-1-063 + 0-007] - -0-02112. 

This is written in on the last line but two, and the approximate value of 
2y(Q-4) = 0-71089 is built up from it. The values y '(0 1),;y '(0-4) are calculated from 
this value of 2 y, and then the integration carried out, giving 2y(0-4) 0-71 US i. 

The difference of this value from the trial value 0-71989 is not such as to affect y or 
y" to the accuracy to which they are kept. The integration has now been taken to 
x = 0-4, and the sequence of operations can now be repeated for the next interval. 
Notes: (i) Since with the interval taken the values of 8y" are divided by 00 before 
being added to those of y', it is adequate to keep y" to one decimal fewer than 

(ii) It would bo practicable to keep another decimal in 2 y without using a 
smaller interval 8x. 

7.31. Another method for a first-order equation 

If in the equation in the form >/' = f(x,y), the function / is not such 
that y" can be obtained to an accuracy adequate lor use in the method 
of the preceding section, a formula involving only y and its differences 
can be used. The appropriate central -difference lormula, w ith an error 
term of the same order as that of (7.10). is 

2/1—2/o = 2 ( 8 -0[.'/o + y'l - i'atf Vo + 8 Vi)] • < 7 • 1 1 > 

But this is not so convenient, because 8-y\ involves y.,, which is not 
known until the next interval, from y x to y.,, has been completed, and 
because the error term is larger than that ol formula (7.10). 

When the integration has reached .r„, only //„, //«. mid backward 
differences from y 0 are available. There is an integration formula u-ing 
backward differences from the beginning ol the interval, but it is not 
satisfactory for practical use since the coefficients of the neglected higher 
order differences are so large; that of the fourth difference is instead of 
7 y 0 for formula (7.10). Kven if the trape/.oidal approximation is used it 
is necessary to estimate y x to give y\, and to adjust this estimate until 
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it agrees with the result of integration with the corresponding value of 
y' x ; and without the value of y" 0 it is more difficult to obtain a good 
approximation for the first estimate. Even when y\ is known, also 
has to be estimated, and this estimate cannot be confirmed until the 
next interval has been completed. An examination of the differences of 
y and of y in the worked example of the previous section will show the 
advantage of having values of y" available so that the only estimate that 
has to be made is one of 8 2 y". 

7.32. Second-order equation with the first derivative present 

The most convenient practical treatment of a second-order equation 
with the first derivative present depends on the form of the equation. 
For a linear equation 

y” +f{x)y'+g{ x )y = H*) ( 7 * 12 ) 

the terms in y' can be eliminated by the use of 

Y = 2 /exp[£ J f{x) rfxj; 

this gives 

y'+tiW-WW-ttfMW = AMexp[l J7(x) dx\ (7.13) 

which reduces the equation to tlie form treated in §7.2. This is likely 
to be a convenient reduction for the homogeneous equation, in which 
/,(. r ) =- 0. I f h(x) is not zero, the exponential factor may make the right- 
hand side of (7. 1») vary too rapidly to be convenient for numerical work; 
though since )' has to be divided by a corresponding exponential factor 
to give I he solution y required, it may be possible to drop the less signifi- 
i ant digits of Y as the solution proceeds. 

If the equation is linear in y', though not in y: 

y"+f (*)!/'+g{*>y) = ( 7 - 14 ) 

the term in //' can be eliminated by the same change of variable, though 
the resulting equation is not so convenient as (7.14). If, however, g(x, y) 
is periodic in x and a periodic solution of y is required, this reduction 
of the equation is not very convenient, as the function Y will not in 
general be periodic. Then it is probably best to use the equation in the 
form (7.14). 

A general method of treating the general second-order equation 

»/’+/(•**, */,y') = 0 (7.15) 

is to regard it as two simultaneous first-order equations 

y'= z, ~-\-f(x,y,z) — 0, 
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the latter being integrated first in each interval. That is, y is first 
integrated to give y' , and then y is integrated to give y. The value of y 
at the end of the interval can be estimated by use of 

S 2 y 0 = (&r) I [l/o+i l 2 & 2 '/o] 

of which only the term J.S 2 yj has to be estimated, and for the integra¬ 
tion of y' to give y , y\ is known so that the integration formula (7.8)^can 
be used for this integration. If the function/fir, y,y') in equation (7.1.)) 
is such that y'“ can be evaluated to adequate accuracy from values of 
x,y, and y', then the method of §7.3 can also be used for the integration 
of y" to give y'. For example, for the van der Pol equation 

y'-(i-3/V+*y = 0 

we have y"'-(l -yV+W> 2 +*2/' = 0 

and can use the method of § 7.3 twice in each interval, once to integrate 
y" to give y' and then to integrate y' to give y. This reduction of a second- 
order equation to two first-order equations should no! generally be used 
for a second-order equation with the first derivative absent. 

7.33. Equations of order higher than the second 

If it is required to treat numerically an equation of order higher than 
the second, it is best to break down the integration through each interval 
into a sequence of single and twofold integrations. In each interval the 
highest derivative should be integrated first, and the lower-order 
derivatives in succession; then, apart perhaps from the integration of 
the highest-order derivatives, formula (7.9) can be used for any single 

integration required. 

7.4. Taylor series methodf 

There is another method, which is in principle applicable to equations, 
of suitable form, of any order. Its limitation is that it is only su.ta lie 
for equations in which the relation between the derivatives is gnen >\ 
an analytical formula, so that it can be differentiated formally as many 
times as is required. For example, in the case of the equation 

y' = x 2 --y 2 

we have in succession 

y" = 2(x—yy'), y” = 2[1—{yy'-Hy') 2 }] ). (7.17) 

y(r= -2[yy'"+3y'y"], y” = -2[yy <, ’+4y'y'"+3(y') 2 ] etc. 

It would be possible here to substitute for y' from (7.16) in the first ot 

t Seo, for oxamplo, J. C. P. Miller, Bri.M A„ or, Worn Ma.Umaiicat Tables, Par,, 
volume li. The Airy Integral (1940), Introduction. § o. 
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equations (7.17) before differentiating, but this would lead to more 
complicated formulae, and it is better to carry out the substitution 
numerically rather than algebraically. 

Consider first a first-order equation such as (7.16), and suppose that 
the solution has been taken to x = x 0 , so that y Q is known. Then y” Q , yj' , 
y x 0 v ,... can be calculated in turn from a set of relations such as (7.17), and 
then y l can be calculated from the Taylor series 

Vi = y{* o+hz) = y 0 +(8^o + ^(M 2 2/o + ^( 5 ^) 3 yo , +---- 

It is convenient to arrange the numerical work so that the terms con¬ 
taining odd powers of Sx and those containing even powers are added up 
separately: 

Seven = Uo + ^ (M^O + ^ (8*) 6 

S od d = @ x )y'o + ^ (8z) 3 i/o" + ^ (S*) 5 yS+... 

Th en y x = S CVCQ +S odd j 

and V-1 = S cven —<S odd I 

This calculation of y_ v the starting-point for the previous interval, 
from y and its derivatives at x = x 0 is a very good check; y 0 has been 
calculated from y and its derivatives at x_j, so that almost all the numbers 
involved in the calculation of y_ x from y 0 by (7.19) are different from 
those involved in the original calculation of y 0 from y_ v 

There is no particular reason for working with the derivatives them¬ 
selves rather than with convenient multiples of them. In this case the 
convenient multiples are the quantities Y in) defined by 

^ (8.r)V'"; 

these are sometimes called ‘reduced derivatives’. That is, 

y<°) = yt ym = ( 8x)y', Y™ = ?,(8.r)V, .... 

Then, for example, (7.16), (7.17) become 

}*n> = (S.r)[.r‘ i — (}’<°)} 2 J, 

T< 2) = (8a:)[.r8.r 

I'«3) ^ J(8x)[(S.r) 2 — 2y(°>I , ^-()'U)}2J, 

Y w = - > (Sx)[r<o>r*)-l_ etc., 
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(7.20) 


and (7.18) becomes 

V _ y<o)iy<2> i v(4) i y(6) i 

*\vcu — ~ 1 o > -* o i 1 o i ••• 

*\tld — i 0^-‘0 ' J 0 a ••• 

No special procedure is necessary for starting the integration. 

For a second-order equation y\ has to be calculated from a Taylor 
series as well as y x . We have 


y± i = !/o± (& x )l/o + ^ fi x ) 2 y 0 » 


and hence 


Y"\ = (5 x)y' ±1 = n»±2r« ) +3y? , ±4yj ) 4, +...; 
so that if we write 

s;„,, = 2 y {, 2 >+41«>+G >'<„*>+- - -. S'n.w = n i, +3Jt’+ 5l ’'o 5, -f--. 

the reduced first derivative at x = x, is 

)•L" = s;. r ,., 1 +s;, 1J 

and the check on the integration is provided by 

Y"\ = >s; >dd -s;. vcn . 

By taking the series (7.20) to several terms, it is practicable to make the 
truncation error of considerably higher order in (8s) than it is m the 
case of formula (7.9) or (7.11). and so to work with a larger interval 8s 
or alternatively to a greater number of significant figures. Results to a 
large number of figures will probably not be required except foi equations 
which do satisfy the conditions for this method to be practicable, and 
in such cases it is a very powerful method. 

7.5. Other procedures 

A number of other procedures have been proposed for the numerical 
integration of differential equations. A few will be summarized in the 

following sections. 

7.51. Richardson’s ‘deferred approach to the limit’ 

In the procedures so far explained it has been the purpose to make 
each interval of the integration correct, within the tolerance for rounding 
error, before going on to the next. This is done by keeping the 1 1 unc.it ion 
error in each interval less than the rounding error. An alternative 
procedure is to carry out a whole integral ion using a \ ay simph inti gi a 
tion formula for which the truncation error is greater than the rounding 
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error, and only correcting for the truncation error after the whole integra¬ 
tion is completed. Such a process has been called by L. F. Richardsonf 
a ‘deferred approach to the limit’; a process of this kind is applicable to 
quadrature as well as to the numerical integration of differential equa¬ 
tions. 

If in integrating the first-order equation 

y' = /(*, y) 

we use simply the trapezoidal formula 

ty* = yi-yo = i(&*0(/o+/i)> ( 7 - 21 ) 

the result y at a given value of x will depend on the interval length x 
used in the integration as well as on x. Let us express this by writing 
this result as y{x,hx)\ the solution of the differential equation is the 
limit of this as Sx -> 0, namely y(x, 0). 

Now in each interval the error in by calculated by (7.21) is of order 
(S.r) 3 . The number of intervals required to cover a given range of x is 
inversely proportional to S.r; hence the aggregate truncation error is of 
order (S.r) 2 . Such an error in y results in an error in y' of order (Sx) 2 , 
which makes an addit ional error of order (8a;) 3 in each by, which is of the 
same order as the truncation error in that interval alone. Thus the 
aggregate error at any given x is of order (Sa-) 2 . 

if now two separate integrations are carried out, using the same 
integration formula (7.21), with different interval lengths (S.r), then 
the leading term in the aggregate truncation error can be eliminated by 
extrapolating to S.r = 0, linearly in (S.r) 2 , at each value of x. The most 
convenient way of doing this in practice is by use of one set of intervals 
bx - h and another set S.r = 2/?, the process is represented graphically 
in Fig. I 2; a convenient numerical process is represented by the formula 

y(.r,0) = y(x.h)-][y(x,'2h)-y(x,h)]. (7.22) 

This process has been called by Richardson ‘/^-extrapolation’. It is 
important to ensure that cumulative rounding errors do not vitiate this 
extrapolation to S.r = 0. 

It is in principle possible to carry out this process of extrapolation to 
S.r = 0 from results calculated for more than two different interval 
lengths, but this is not a satisfactory procedure in many cases. 


t L. V. Ruhanlson, Phil. Trans. Rot/. Soc. 226 (1927), 300. 
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7.52. Iterative processes 
The solution of the equation 

y' =*f{x,y)> !/(- r o) = 

can formally be obtained by const ructing a sequence of function y Ut) {x) by 

y ( „ +1 )(^= y 0 + f /{•*•>y(„>(*)} dx. (7.23) 




If the integral is evaluated by quadrature, this process of iterative 
quadrature is the numerical equivalent of Picard’s process in the formal 
analytical theory of differential equations. It is sometimes useful for 
starting an integration, but unless a good approximation is available to 
use as a first approximation //„(.<•) in the right-hand side of (7.23), it is 
seldom useful for carrying the integration of an equation with one-point 
boundary conditions over a range of x, because the convergence of the 
successive functions to the solution of the equation is too slow. A 
form of iterative quadrature may, however, he useful in dealing with 

problems with two-point boundary conditions (see §7.(i). 

As an example of another, more practical, kind of iterative process, 

consider the equation lf " __ (7.24) 

A sequence of functions y {ll) can be formed b\ 

8 2 y ( , () = (S-t‘)"[/{- r ’ 2/(»i)( a ’)} - 1) — 240^ 4 //(n-X)]' 

Here only the leading term on the right-hand side depends on the 
function y (ll) currently being evaluated; the ‘correcting’ terms, involving 
hhj\ h*ij" are derived from the previously calculated function y {n -w 
This avoids any estimation of &’//", and enables h*y" and 8°»/". and 
higher differences, to be included. 


(7.25) 
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A different iterative process is obtained if y" on the left-hand side of 
(7.24) is expressed in terms of y and its differences. 

From formula (4.43) we have 

(M 2 Z/o = 3'h/o- £ ,Jo 88 '/o+ 


so that if y satisfies equation (7.24), 

Vy = (8x)V(x.y)+ l yV-tf^+6WV+0(&r)i°. (7.20) 

Then a sequence of functions y {ll) can be formed by means of the iterative 

formula *,.« ■= (Wk.#.«il+i5'*.ry 4 fa+OWJ, (7-27) 


where a 2 is the twofold sum operator as in § 7.22. Here the ‘correcting’ 
terms in the evaluation of y lu . j) are expressed in terms of the differences 
of the previous function y (ll) itself, instead of in terms of the differences 
of its second derivative. A similar treatment can be applied to first-order 
equations, and various examples of its application have been given by 
Fox and Goodwin.t 

For a linear (but not necessarily homogeneous) equation this process 
can be shortened as follows.J Let the equation be 

V" = /(*)*/+</(*)• (7.28) 

Substitution of [f(x)y+g(x)] for the general function f(x, y) in (7.20) gives 

b-y = (bx)-[f{.v)y +{/(*)] 4- '.&y - 2t)S 6 y+-r 0(8.c) 10 , 
and operation on both sides with (l-f--jh8 2 ) gives 


S 8 Ul“i -A&*)*f)y] - (Sx)*f{x)y+(g+tf* g )-&fi*!!+i&iVy+Otfxr. 

(7.29) 


I f t he terms of sixth and higher orders in (S.r) are neglected, this is equa¬ 
tion (7.7). Let c be the solution of this approximate equation, with the 
same initial conditions as those specified for y\ that is, 


and let 


8*[{ 1-^(8*)*/}*] = (8x)*f{x)z+(g+tf*g) t 



(7.30) 


I hen, on subtracting (7.30) from (7.29), it follows that 7 ; satisfies 

S 2 [{> -iW-ZJ,,] - (8.r)y(.td7 7 - 5 i 0 8^+ i ^ 0 8V+0(8a-)io. (7.31) 


From equation (7.31) it follows that 
differs from 8 6 c by terms of order (5.r) 10 . 
with an error of order ( 8 .r) 10 , by 


7 / is of order (8x) 4 , hence 8 6 y 
Hence (7.31) can be replaced, 






+ L. Fox and F.. T. C 

* 'Him troiitvnont is <1 
oi|uation Olver, ibid. 


ood win, I’roc. Camb. Phil. Soc. 45 (104!*). :t7.{. 

11,1 to *'• J- Diver. For a similar treatment of a non-linoar 
40 il'JSO), 570, § 4. 
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Then if 2 is calculated from formula (7.30) and tj from formula (7.32), 
neither of which involve any estimation, the aggregate truncation error 
in y = z+7) is of order (Sx) 9 . 


7.6. Two-point boundary conditions 

As an example of the treatment of two-point boundary conditions, 


consider the solution of 



subject to the conditions 

y = y 0 at x = 0, y = y a at x = a. 

A step-by-step integrat ion, start ing from x = 0, has to start from definite 
numerical values of y at the beginning and end of the first interval, and 
the result at any later value of .r is determined by these two values of y. 
The former is given to be y 0 , but the latter, y{b. r), is not specified. If 
the variation of the solution y(a) at x = a with the value of y(hx) is not 
too rapid, the following process can be used to find the solution satisfying 
the condition at x = a. A set of integrations is carried out for a set of 
trial values of y(5a), and the value of y{a) obtained as a function of y{hx). 
Interpolation (graphical or numerical) can then be used to obtain a 
close approximation, say y v to the value of y(Bx) which gives a solution 
for which y(a) has the required value y a . A further one or two integra¬ 
tions with values of y(8 x) in the immediate neighbourhood of y x then 
enables the solution satisfying the given condition at y = a to be obtained 


by linear interpolation. 

If the value of y(a) is very sensitive to the value of y(8x), such a 
process is not practicable in this simple form. Consider the variation 
of y al a fixed x with the trial value of >/(Sx); this is dy(x)/dy(hx). This 
is a function of x, and, for a non-linear differential equation, corresponds 
to a ‘complementary function’ of a linear differential equation. It might 
behave approximately as e'\ in which case it would increase by a factor 
of about 10 10 over a range of x from 0 to 5. This would mean t hat unless 
the choice of y(8x) were correct to 0 00001, all trial solutions would 
have values of |i/(a)| of the order of 10* or larger. If the condition at 
x = a were y(a) = 1, interpolation between two trial solutions with 
values of y(8x) differing by 0 00001 would determine ?/(5x) closely but 
would not determine the solution at all well except near x = 0. I urther, 
the effects of rounding errors in the early intervals of the integration 
build up in much the same way as the function Zy(x)/dy(Sx), so that it 
would be necessary to keep a large number of figuies, and probabU to 
work to fifteen or twenty decimals. 
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Such a situation is not rare; in the writer’s experience it is more likely 
than not to occur in equations with two-point boundary conditions 
which arise in real problems (as distinct from those which are made up 
to serve as text-book examples). In such a situation, however, a 
procedure of the same kind can be used, proceeding by stages in the x 
direction. Two solutions with different values of y(8:r) are carried to 
such a value of x, say x Vl , that their behaviour indicates clearly enough 
whether the required solution lies between them or not; if not, other 
solutions arc evaluated until a pair is found between which the required 
solution does lie. Let these be y l and y n , with values ^(Sar) and y n {$x) 
at the end of the first interval. 

From the behaviour of the solutions y T and y n at x = ar^, and the 
expected behaviour of the solution required, an estimate is made of 
the fraction 9 of the difference between the functions y l and y n such that 

yi+%n-2/i) 

is a fair approximation to the required solution. Linear interpolation 
will probably not be valid at x = ar u , but should be good enough to give 
one decimal in 9, which is all that is wanted. Another solution. y lu , is 
then started, not from x = 0 but from some value of a'j at which linear 
interpolation between the solution y 1 and y n is valid to the accuracy to 
which the calculation is carried. Whether linear interpolation is valid 
can usually be tested by comparing (a) the value of y n {x l ) interpolated 
linearly between the values for solutions y l and y u , that is, 

and ( h) the value of y"(x i ) calculated from the interpolated value of 
//(.qh If the difference between these two values of y"(a-j) is not enough to 
affect the last digit of >/(.r r fS.r), then, in the usual contexts in which 
this procedure is required, the linear interpolation is adequate. 

According to the behaviour of j/ m , either another solution is started 
from r } . or a solution is started by linear interpolation between y lu and 
.Yn or y| at a point .r in further out, by a repetition of the process for 
selecting and starting the evaluation of the solution i/ m . This process 
may have to be repeated several times before the value x = a is reached. 


7.61. Iterative quadrature 

In 1 he process considered in the previous section the solution satisfying 
Uie two point boundary conditions is reached by evaluating a sequence 
unctions each ol which dors satisfy the differential equation but 
,,( d sutisly all the boundary conditions. An alternative procedure 


ot 
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in some cases is to approach the solution required through a sequence oi 
functions each of which satisfies all the boundary conditions, but does 
not satisfy the equation. 

Consider, for example, the equation 


y" = -(i W~)y" 

with boundary conditions 

y = 0, y' = 0 at a; = 0, y -> 1 as .r -> x. 

Let 2 / (m) (x) be a sequence of funct ions defined by 

//(«+1) = (1 "i"2/</i))2/<n +1)’ 

If at any stage of the work y (ll) is a known function of .r, this is an equat ion 
for the next function of the sequence, namely y u + it is linear and 
homogeneous in this unknown function and there is no difficulty m 
obtaining a solution of (7.35) satisfying all three of the boundary condi¬ 
tions (7.34). One integration gives 

»('„+.)= exp [-J < 1 +»?„,) d* 

6 

where A is, so far, an undetermined integration constant. Another 
integration gives 

»;.+« = 4 / ex p{- / (H dx; 

6 1 o 

the condition y'{ 0) = 0 has been satisfied by choosing the lower limit of 
the integral; the condition ,/(x) = 1 can now be satisfied by choice of 

A, and gives . i r . „ , . ) . 

f exj)j — I (1 4* yfn)) dxj dx 

, _t 6_ 

2/(n + 1 ) «-■ 


(7.33) 


(7.34) 


(7.35) 


f expj— | (1 d"yin)) j dx 

O * 6 


and another integration from lower limit * = 0 satisfies the condition 
•on ,(0), giving . ^ l+yU)ix y xix 

n A V (I _ _ 

ji 


fKl 

Uin 4 1) 


(7.3(5) 


(" expj— I* (1 +!/?«)) 'kj 
6 ' » 

This may appear a rather elaborate form of equation (7..L>), but <■ '' 
tains the boundary conditions (7.34) in addition, and is m fact quite 

convenient for numerical work. 

6363 L 
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Unless y in+l ) = V( n ) to the accuracy of the numerical work, y (ft+1 ) is 
not a solution of equation (7.33), so that the separate members of the 
sequence y(„){z) are not solutions of the equation though they do satisfy 
all the boundary conditions. But if the process converges, in a numerical 
sense that after a finite number of repetitions of the iterative process 
2/(n+i) becomes equal to y M to the accuracy to which the numerical work 
is taken, then to this accuracy such a function y(„ +1 ) is a solution of the 
equation (7.33). 

A process of this kind, when available, is particularly useful in cases 
in which, using a step-by-step integration, y(a) is very sensitive to y(Sar). 
This sensitiveness is an indication of a kind of instability in the step-by- 
step process; but this instability does not correspond to any instability 
in the physical system to which the equation refers, just because in the 
physical system the behaviour of the system is fixed at both ends of the 
range. It does not appear either in a treatment such as that of the present 
section, which approaches the physical situation more closely in that it 
insists at each stage that the approximate solution is tied at both ends, 
however it behaves intermediately. 

Another method, which also works through a sequence of approximate 
solutions satisfying the boundary conditions at both ends of the range, 
is an application of the relaxation method’ of Southwell (see § 8.5 and 
end of § 8.53). 

7.62. Characteristic value problems 

In the equations considered in §§7.6 and 7.61, the parameter which 
was adjustable to fit the two-point boundary conditions was y(8ar); 
variations of this are equivalent to variations in y' Q , since for a second- 
order equation y{hx) is determined in terms of y 0 and y Q (compare § 7.4). 
So that in effect the adjustable parameter is one of the initial con¬ 
ditions. 

In another important class of problems the adjustable parameter is 
a constant in the equation itself, for example the constant A in the 
equation 

Z/" + lA-i-/(.r)]y = 0 (7.37) 

with boundary conditions 

.'/(«) = y(b) = 0 (7.38) 

and the further condition that the solution is not identically zero. The 
values of A for which the equation (7.37) has such solutions are called 
the characteristic values of A. Depending on the form of f(x) and the 
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values of a and 6, there may be one or more discrete values of A and 
perhaps a continuous range. For example, for the equation 


y" + = o 


with the condition that y should be 0 at x = 0 and finite at x = co, there 
is an infinite set of discrete characteristic values A = — 1/n 2 (n integral) 
and a continuous set A > 0. 

The determination of characteristic values and corresponding solutions 
can be carried out by evaluating trial solutions with trial values of A, 
using a process very similar to that of § 7.6.| 

For many forms of the function f{x) for which solutions of (7.37) are 
wanted in practice, there is a finite least value of A; this can often be 
determined approximately by using Rayleigh’s principle .% This states 
that if z is an approximation to the solution of (7.37) for the lowest 

value of A, then z, . ft 

J z{z"+f{x)z) dx J z- dx (7.39) 

a a 

ft 

differs from A by a quantity of order j (y-z) 2 dx, so that a rough estimate 

a 

of z substituted in (7.39) gives a fair value of A. I his may often give a 
good first trial value to use in the numerical integration of equation 
(7.37). 


7.7. Further notes on numerical integration of ordinary differen¬ 
tial equations 

With a first-order linear equation, y'+f{x)y = g(x), there is a strong 
temptation to follow the standard text-book method and reduce the 
solution of the equation to a quadrature by use of an integrating factor. 
There are occasions for which this is useful (see § 7.01 for example). But 
the writer’s experience has been that in the great majority of cases arising 
for solution in practice, this is a temptation to be resisted, and that it is 
considerably easier to evaluate the solution by numerical integration 
of the equation as it stands than to evaluate the integrals in the solution 

in quadrature. An example has been given in § 1.1. 

On the other hand, in a second-order equation linear in the first 
derivative, it may be worth while making a change of variable to remove 
the term involving the first derivative, especially if the equation is 
linear and homogeneous. This has been considered in §7.32. 

t For anothor jirocoiluro, soo § 8.7. Sou also W. E. Miluo, Joum. oj Research of Sat. Bur. 
of Standards, 45 (ljJ.'.O). 245. 

J Soo G. Toinplo and W. G. Bickloy, Ka'jleiyh'e I'rincipU (Oxford, 1933). 
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7.71. The Madelung transformation 

For evaluating oscillatory solutions of a second-order homogeneous 
linear equation with the first derivative absent: 

y"+f{x)y = 0, (7.40) 

a transformation due to Madelungf is sometimes useful. 

Let us write y = F(x)exp^ J g{z) cZ-rJ, (7.41) 

where F and g are to be real; this is equivalent to describing the oscil¬ 
lating function y at each point in terms of a local amplitude F(z) and 
local phase ) = | g(z) dz. The real and imaginary parts of (7.41) 
clearly give two linearly independent solutions of (7.40). Substitution 
of (7.41) into (7.40) and separation of real and imaginary parts gives 

F"-Fg°-+fF = 0, (7.42) 

2 F'g+Fg' = 0. (7.43) 

The second of these two equations is integrable and gives 

F'-g = const. — .4 (say), (7.44) 

and substitution in (7.42) then gives 

F”—A*lF*+fF = 0. (7.45) 

Thus the evaluation of the two functions F and g can be separated, 
equation (7.45) being first solved for F, and g then determined. 

Equation (7.45) is not linear, but it may be easier to integrate than 
the original equation (7.40), since the raj)idly oscillating part of // lias 
been taken out by the factor expfi J g{x) and the function F de¬ 
scribing the local amplitude of this oscillation will often vary relatively 
slowlv.l 


7.72. The Kiccati transformation 

l'or equation (7.40) with/(*) negative, the Kiccati transformation 

V = \i U = d[\ogy)jdx, V-f T+f(z) = 0 (7.40) 

i- sometimes useful, especially if f(z) is negative over a considerable 
ri'nge ot z. It /(.r) is negative and slowly varying, and '/(•*')j is large, a 
solution ot equation (7.40) can sometimes be obtained bv writing it in 

V = ±[-f{x)-v’Y (7.47) 


he form 


t !;. Zt'i/.fur Pl.ys. 67 (11*31), M0. 

; I ui an «.'\aiiiplt* <>1 the use of this transformation, seo D. R. Hart re©, U. L. Kronig, 
H. PuiJurson, 1 (11/3-1). 81*5. 



INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS 149 
and solving this by iteration. The first approximation is 

v = [-/<*)]*; 

• 

numerical differentiation then gives an approximation to V in (7.47) and 
hence a better value of rj. This is one of the few situations in which 
numerical differentiation may be useful as a tool in a practical numerical 
process. 


I 



VIII 

SIMULTANEOUS LINEAR ALGEBRAIC 
EQUATIONS AND MATRICES 


8.1. Direct and indirect methods for simultaneous linear 

equations 

The necessary and sufficient condition that a system of linear simul¬ 
taneous equations should have a solution, and that the solution should 
be unique, is that the determinant of the coefficients should be non-zero. 
In this chapter, except in § 8.C, it will be assumed that this condition is 
satisfied, so that we shall only be concerned with the determination of a 
solution when a unique solution exists. 

Any text-book of algebra shows that the solution of such a set of 
equations can be expressed in terms of ratios of determinants, and one 
w r ay of evaluating the solution of the equations would be to evaluate 
these determinants numerically. But though there may be no one best 
way of evaluating the solution, it can be said with some certainty that 
the direct evaluation of the determinants and of the expression for the 
solution in terms of them is never the best way, though of course the 
evaluation of a solution by any other method must come in the end to 
t he same thing as the evaluation of the solution in terms of determinants. 

The general form of a set of such equations will be written 


or shortly 


«ii*i+ff 12 a- 2 +a 13 r3-f... = b y 

^21 = bo, CtC. 

2 a H x i = b { (i = 1 , 2 ,..., n), 

) 


or in matrix form 




The equation V a tJ Xj = 6, with any particular value of i will be called 
the Tth equation' and n will be used throughout for the number of 
equations and of unknowns. I will be written for the unit matrix, that 

is the matrix ot which all the diagonal elements are 1 and all the non¬ 
diagonal elements 0. 

'There is probably no one way for evaluating the solution of such a set 
of equations which can be said to be the best in all circumstances. 
The most effective practical method to use depends on various character¬ 
istics of the equations and of the solutions required and on the experience 
of the individual who has the task of carrying out the numerical work. 
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The character of the equations is concerned with such matters as whether 
the coefficients are all small integers or not, whether many are zero, and 
whether those which are non-zero are arranged in some systematic way, 
whether they are all exactly known or are subject to uncertainty as a 
consequence of being either experimental measures or results of other 
calculations subject to rounding errors, and whether the diagonal 
coefficients are large compared with the non-diagonal coefficients or 
not. 

Relevant characteristics of the solutions required are whether a 
solution is wanted for one set of values of the 6 ; ’s only or for many such 
sets, and whether the characteristic values of the matrix A are required 

as well as the solution of the equation. 

There are two main kinds of method, sometimes called direct and 

‘indirect*. 

‘Direct* methods are those in which one application of the computing 
procedure leads to the solution, to an accuracy depending on the nominal 
accuracy of the calculations. The evaluation of the expression for the 
solution in terms of determinants, and the method of elimination, are 
examples. 

‘Indirect’ methods are those in which the solution is approached by 
successive approximation, by a number of repetitions of the same com¬ 
puting procedure. For hand calculations an advantage, when such a 
method is applicable, is that in the early stages only a limited number 
of figures need be kept, and the accuracy increased as the solution is 

approached. 

If an approximation x l = = £»,.••> in general x, = £ it to the 

solution of the equations has been obtained, the quantities 


Ri= I a uZj- b < 


( 8 . 2 ) 


are called the ‘residuals’ of the various equations; they measure the 
extent to which the approximate solution x, = fails to satisfy the 
equations. The corrections A& = (*,-&> to the approximate solution 

satisfy the equations £ a Af = c t = — 7?, (8.3) 

i 

with the same coefficients as the original equations. In \ arious methods 
of evaluating a solution it is possible to proceed by a process of approxi 
mation, obtaining first an approximate solution, calculating t le 
residuals for this approximate solution, then solving (8.3) foi <om< tions 
to the approximate solution, and perhaps repeating this process. 
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8.11. Matrices 


If, as assumed, the determinant of the coefficients is non-zero then 
the matrix A of the coefficients has an inverse A -1 , and the solution is 

formall y x _ A -1 b. (8.4) 


If many solutions for different sets of values b { are wanted, then it may 
be worth while to evaluate A -1 first and then to evaluate solutions 
from (8.4); one process for the inversion of a matrix is considered in 
§8.3 and another in §8.41. In general the calculation of A -1 will be 
affected by rounding errors, so the matrix used as A -1 in evaluating a 
solution from (8.4) will not be exactly' the inverse of the matrix A of 
the coefficients of the original equations. The results of evaluating 
(8.4) with this approximate A -1 may therefore have to be treated as an 
approximation % to the solution and improved as explained at the end 
of the previous section. 

A matrix in which all elements below the diagonal are zero is called 
an ‘upper triangular’ matrix and one in which all the elements above the 
diagonal are zero is called ‘lower triangular'. The determinant of a 
triangular matrix is the product of its diagonal elements. 


8.12. Ill-conditioned equations 

If the determinant D of the coefficients is expanded in terms of the 
elements of any one row or column, for example 


D — J A Jk a Jk 

k 


(8.5) 


(the sum being over k only, j being fixed), it may happen that D is small 
compared with some of the individual terms in this sum. Then the 
\ alue of D, and so the solution of the equations, is very sensitive to small 
changes in the values of the coefficients, and an approximate solution 
obtained by a numerical method which is subject to rounding errors is 
likely to be very sensitive to these errors. If, for example, D is of the 
order of unity but some of the individual terms in (8.5) are of the order 
of 2000. then a change of 0-1 per cent, in the coefficient. a jk in one of these 
laige terms may change the value of D from -f 1 to — 1, and the solutions 
in the two cases, and for intermediate values of this coefficient a jk , may 

he entirelv different. 

% 

A 'civ elementary example is provided by the equations 

■ rJ r--!/ = 4, lOOO.r-f 2001?/ = 4003 
for which /> 2001-2000 = 1, and the solution is 


EQUATIONS AND MATRICES 


1 ">3 


If the coefficient of y in the second of these equations is changed by 
— 01 per cent, to 1999, the solution becomes 

x = 10, y = —3 

and if it is changed by +0-1 per cent, to 2003, the solution becomes 


* = 2 , 


y=\. 


If the coefficients in these equations are known exactly , then the solution 
can be determined to any accuracy required. But if the coefficients are 
subject to some uncertainty, either through being derived from observa¬ 
tions which can only be made to a finite degree of accuracy, or throug i 
being themselves results of other calculations which may be alfectec 
by rounding errors, then clearly not even the sign and first significant 
figure of the solution can be determined unless the uncertainties in the 
coefficients are less than 01 per cent, A set of equations for which D is 
small compared with some of the individual terms in the sum (8.o) is 

called‘ill-conditioned’. 

It is sometimes said that a set of equations ,s dl-cond.t.oned ,f the 
determinant D of the coefficients is small; hut this is an inadequate 
statement because the relevant standard of smallness ,s unspecified. 

Consider a set of thirty equations for which D - . »<? r j' * lo,ls 

between the variables expressed by the equat ions are not altered if each 
equation is multiplied by 1000; but 1) then becomes 10 which is not 

‘small’ in any ordinary use of the word. 

‘Ill-conditioned’, applied to a set of equations ,s some,,,™ »od 
merely as a qualitative term of abuse; but it is capable of being given a 

quantitative significance. Let A-, A 12 '.A"" be the eharac.er, s t,e vahr^ 

of the matrix of the coefficients a jk , and let ,a , I f . . 

and least of the quantities |A-|; the„_|W .3 “ " 

measure of ill-conditioncdncss; when this ratio is ncai > • 

equat ions are well-conditioned, when it is large compared with thea e 
id-conditioned; when At- is aero the determinant D of the c , c 
is aero and either the equations have no solution or the solution 

“"unfortunately the numerical determination ofIbis ™ 

conditionedness is as long a process as the evaluation of the ■ d 

the equations, and the same applies to some other 'measures of condition 

which have been suggested by Turing, t 111S not ' 11 ust 

t A. M. Turing, Quart. J. itrek. «W A r ,M Math. 1 (IMS). 287 - 
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for giving advance warning that the equations concerned are ill-condi¬ 
tioned. In many cases the intermediate results obtained in the course 
of the numerical process of solving the equations exhibit characteristic 
symptoms when the equations are ill-conditioned. These symptoms 
depend on the particular process and will be mentioned in the course 
of consideration of the individual processes. 

Sometimes inspection of a set of equations will suggest that they are 
ill-conditioned. Expressed geometrically, any one of the equations is 
the equation of a hyper-plane in rc-dimensional space, and the coefficients 
in the equation are the components of a vector normal to this hyper¬ 
plane. If these normals are all in much the same direction then the hyper¬ 
planes are nearly parallel so^hat their intersections are at very acute 
angles, and the common point of them all, which represents the solution 
of the equations, is not well determined. For example, it is clear on 
inspection that for the equations (constructed by T. S. Wilson and quoted 
by J. Morris)f 

5*1 + 7* 2 +6* 3 + 5x 4 = 23 'j 
7.*.,+10 x 2 +8x 3 +7x 4 = 32 
6x,+ 8*0+10*3 +9x 4 = 33 
5*i + 7* 2 +9*3+10* 4 = 31 J 


the normals to the hyper-planes make only small angles with each other, 
so that this set of equations is ill-conditioned. But it is only occasionally 
that an ill-conditioned set of equations can be recognized as such by 

inspection. 

A characteristic feature of ill-conditioned equations is that a set of 
values for the unknowns which differs considerably from the solution 
of the equations may, nevertheless, give small residuals for all the equa¬ 
tions. For example, for the equations (8.6) the residuals for certain sets 
of values of * 1 ,* 2 ,* 3 ,* 4 are as follows: 


-'i 

f 14 0 


— 7-2 


_O. r 

— •- o 


^4 

+ 31 
+ 1-21 


■ 2-30 +018 +0-05 +1-21 «! = -S = -«! = = 0-01 

" heroas ,hc exact solution is r, = ,r 2 = r 3 = x 4 = 1. Thus in this 
cas.- values of the residuals which are less than 1/2000 of the values of 
the bs in the equations still do not guarantee the accuracy even of the 
first figure in the *’s. This is an extreme case, but it illustrates the need 

l ’ r " a,,hon 1,1 taki »g the smallness of residuals as a guide to the accuracy 
° tlie s °hition when the equations are ill-conditioned. 

+ J. Morris, I'hil. Mug. (7) 37 (1946), 106. 
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8.13. Normal equations 

For a set of values.of (x„r 2 .*„), not necessarily a solution of the 

equations, let 2 S be written for the sum of the squares of the residuals 

(8.2), that is, S = i 2 *!• 

i 

Then S is a quadratic function of (*„**.-.*«) which is zer0 for those 
values of (x v .r 2 ,.which form the solution of the equations, and is 

positive for all other sets of values. Hence the determination of the 

solution of these equations is equivalent to finding the set of values of 

\x v x 2 ,...,x n ) which make S a minimum. This set is given by 

dS = 0 (all j) 


dXj 




or 


(8.7) 


Now dRJdXj = a i} - {a) jit 

where A is the transpose of the matrix A. Hence the set of equations 
(8.7) is J.d j{ Ri = 0 

( 8 . 8 ) 


or 


2 [{3ji a tk)x k — s ji b i\ ~ 0 

iK 


or in matrix form (AA)x Ab 0. 

The set of equations (8.8) is sometimes called the set °f ‘normal 
equations’ corresponding to the original equations (8.1 They are 
derived from the positive definite quadratic form S, anil tin matrix 
(AA) is necessarily symmetrical; these features are advantageous m 
some methods of carrying out the solution numerically. 

On the other hand, the ratio the normal equation s 

greater than that for the original equations.t so that the l ^ 

lions are less well-conditioned than the original equations, and on‘he 
original equations are at all severely ill-condit ,oned the normal q.m ions 
are very much worse. Hence, methods winch do not depend on the 

matrix of the coefficients being symmetrical, or on thc 

derived from a positive definite quadratic form, are often pichi.ibk. 

8.2 Elimination . ,. ■ 

A straightforward direct method is one based on d 

tion of the variables. This is a development oft he process of, m ■ > 
algebra with attention given to three points, namely systematic 

+ Sc-e Olgu TuuBHky. M.T.A.C. 4 (M»5U). 111. 
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arrangement of the work, provision of a current check, and control of 
rounding errors. The points are all important and become more impor¬ 
tant the greater the number of equations and unknowns. 

In elementary algebra emphasis is properly given to the importance 
of verifying that an alleged solution does actually satisfy the equations. 
This final check should always be carried out. But in the evaluation of 
the solution of a set of more than three equations it is hardly sufficient 
since if it fails it gives no indication of the location of the mistake, and 
the whole work has to be repeated with an appreciable probability of 
repeating the same mistake. A current check is required, both to help 
in locating a mistake and to prevent much further work being based on 
erroneous intermediate results. 

To eliminate x k between the ith and jth equations, in which the 
coefficients of x k are a ik and a jk respectively, we have to multiply the 
equations by olj and a, respectively, and add, choosing olj so that 

oc j a ik +<x i a jk = 0. 


Formally, there is an infinite number of ways of choosing the multipliers 
and <xj\ but in practice there are only two ways which are generally 
useful. One is to take one of the multipliers as unity, that is to say to take 


or 

the other is to take 


a, = 1, ocj = —a Jk fa ik } 

n j — * > a i = ~ n ik ' a jk I 

«j ~ ± n jk> = ^ifr- 


(8.9) 

( 8 . 10 ) 


The division involved in the choice represented by (8.9) will usually 
involve rounding errors in each elimination. Since the results of eliminat¬ 
ing one variable arc used later in the elimination of other variables, it is 
important to keep these rounding errors under control, and this is best 
done as follows. One of the multipliers is taken as unity; we choose that 


one of the alternatives (8.9) which makes the modulus of the other 

multiplier less than unity. Then at each stage of the elimination the 

rounding errors from previous stages are always multiplied by numbers 

of modulus less than unitv. 

* 

This is a general method. If, however, the coefficients are all fairly 
small integers, it is possible to carryout the elimination without intro¬ 
ducing any rounding errors, by using the choice (8.10) of the multipliers, 
which avoids any division. If this can be done without the coefficients 
becoming inconveniently large it is probably the preferable choice. It 
may be possible to keep the coefficients from becoming large bv making 
suitable linear combinations of the equations, with integral coefficients, 
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in the course of the elimination process; since no rounding off is involved 
there is no loss of numerical accuracy in such a procedure. As a simple 

example, with the equations 

(1) 2&r+31y = &j, (-) 44.r+65y = b 2 

one should not try to eliminate .r by multiplying the first equation by 
44 and the second by -23 and adding; the first step should be to form 

the linear combination 2 x (1) (-)'• 

(3) = 2 X (1)—(-)> 2u:-3y = 2 b x -b 2t 

before continuing the elimination. 

8 .21. General elimination process 

To eliminate the variable t, from a set of equations, specified by 
different values off, we may take one equation, say thejth, and foi each 

talue of i form ^ cll , ia ti on )-f (*,) X (jth equation), (tU1) 

where a,- is given by the seeond of formulae (8.0). If the numerical work 
is suitably arranged, it is not necessary to write out the equat.ons n full 
at each stage. It is enough to write down, for each equat.on, Ire co¬ 
efficients and the constant term in appropriate columns. A current cheek 
can be provided by keeping a record, with each cquaUcn. of th of 
the coefficients and the constant term, and form.ng he linear comb ^ 
tion expressed by (8.11) not only for the coeffie.ents but .L» fo Ins 
check sum. The value of this check sum for the .th equat.on a,11 be 

written s t ; that is s _ __ ^ a it . 

Thus for using the jth equation to eliminate r, fro... the /th equation, 
we have the following scheme: 

Check 

tin m 

.s 


ith equation 

Coefficients of 

*k x * 

« ik (l ‘ l 

Con M<i nt 

term 

t'i 

1) 

jth equation 

“,k 


u ) 

b, -\- X, hj 

- //. 

[*l = ~~ a i k ! a J k ) 


«,•*+ a « w // 

Result 

= 0 

= “it 





N 


s '. f x . ,s '; 

/ 

-- S; 


( 8 . 12 ) 


The check consists in verifying that « f calc ul.itcd .u 


n-t (within the tolerance for rounding errors)^ ht “ 

entries in the corresponding hue, name \ * • y •' 

here is sometimes called the 'pivotal equation*, ami the cocllicicnt u ; , 
which is the divisor in the evaluation ol the cot >< u » *. 
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is called the ‘pivotal coefficient’ or ‘pivot’ for this elimination; it is the 
coefficient, in the pivotal equation, of the variable to be eliminated. 

Example: To use the first of the three equations: 

31-74x 1 + 43-61x 2 -16-94x 3 +16-94x 4 = 41-37, 

6-86xj + 9-81x 2 + 7-68x 3 +3-96.1', = 16-81, 

35-85x x —32-92x 2 +13-81x 3 + 5-94x 4 = 21-84 

to eliminate x 2 from the second and third. 

The pivotal coefficient is the coefficient 43-61 in the first equation, and the 
multipliers are 



«2 = 

— 9-81/43-61 = - 

0-22495, 

a 3 = 

+ 32-92/43-61 = 

0-75487 


Line no. 
and 

operation 


Coefficient of 


Constant 

term 

Check 

sum 

Notes 

*1 

** 


*4 

(1) 


31-74 

43-61 

— 1G-94 

16 94 


116-72 


(2) 


0-86 

9-81 

7-C8 

3-96 


45-12 


(3) 


35-85 

-32-92 

13-81 

5-94 


44-52 


(4) 

= <qx(i) 

-7-140 

- 0-810 

3-811 

- 3-811 

-9-300 

-20-256 

a, = -0-22495 

(5) 

= (2) + (4) 

-0-280 

0 

11-491 

ffgraTTj 

7-504 

18-864 

cross sum 









= 18-864 

(6) 

= a,X(1) 

23-960 

32-920 

12-877 

- 12-877 

31-229 

88-108 

a, = 0-75487 

(7) 

= (3)-HO) 

69-810 

0 

20-687 

- 0-937 

53-069 

132-628 

cross sum 









= 132-629 


Notes: (i) It is convenient to keep a note, on the left-hand side, of the operations 
carried out to obtain the successive lines of the calculation. This simplifies the 
location and correction of mistakes should any be made, and is also useful if the 
calculation has to be repeated with other values of the constant terms 6,. 

(ii) In working, it is best to set each cq on the setting levers or keyboard of a 
machine, t hen first multiply by the coefficient a jk of the variable which it is required 
to eliminate, to check that cq has been set correctly, then multiply by the other 
coefficients in the pivotal equation. These products can bo written down, as in 
(lie above example in lines (4) and (6), or the entries in lines (5) and (7) can bo 
formed and written down directly. 

(iii) The results of elimination of x 2 from the third equation in the above oxample 
illustrate a kind of mistake which is not detected by the check explained above. 

ho check is satisfied but two entries in line (7) are wrong; in line (6) the coefficients 
of .r, and x 4 should read - 12-7S7 and + 12-787 respectively, and the corresponding 
entries in lino (7) need correcting accordingly. The failure of the check to indicate 
the presence of the mistakes arises from the fact that the coefficient ofx 4 in line (6) 
is just copied from t hat of x 3 with a change of sign, and if the latter coefficient is 
wrong t hen so is the former and the effects of the two mistakes cancel in the check. 
Uns shows that special care is necessary when two coefficients in the pivotal 
equation are equal and opposite in sign. It also emphasizes that in the solution 
of a system of equations the current check on t lie eliminations alone is not sufficient; 

1 us current cheek should always be supplemented by a verification that an alleged 
solution uoes satisfy the original equations. 
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8.22. Evaluation of a solution by elimination 

One way of arranging the solution of a system of equations by the 
elimination process is as follows. Use one equation as pivotal equation 
to eliminate one variable from all the equations. It is convenient to 
consider the variables and equations renumbered (if necessary) so that 
the variable eliminated is x v and the equation used as pivotal equation 
at this stage is the first. The result will be a single equation containing 

* i: a n x l +a 12 x 2 +a 13 x z +...+a ln x n = b i (8>13) 

and a system of (n-l) equations not involving x v of which a typical 
one is = (814) 

where a' fl and b\ are given by (8.12) with j = 1. Then another variable 
which can similarly be taken as * t , is eliminated from equations (8.14), and 

If the equation used as the pivotal equation to eliminate .r, is chosen 
so that |a n | ^ |a fl |(i > 1), then none of the multipliers <*, = —(a.,/a.i) 
in this elimination is greater than 1 in magnitude, a condition which we 
have already seen to be desirable to keep control of rounding errors. 
Similarly, the equation used as the pivotal equation to eliminate x 2 

should be chosen so that > I <41 («' > 2), and so on. 

Once an equation has been used as a pivotal equation in the elimina¬ 
tion of one variable in this process, it is left unaltered in the further 
stages of the elimination process. Then this process loads finally to a 
system of equations of which the first is (8.13), the second is that member 
(say with j = 2) of the set (8.14) which is used as the pivotal equation 
for the elimination of x 2 from this set of equations, and so on. This 

SyStem iS .„«,+«„*,+«„*,+«»*.+■■• = b ) 

U 22 X 2 ^1 V — ^2 

a' 33 x 3 +a‘ M x t + ... = *»’ ctc ' 

of which the rath equation contains n-m + l of the variables, and the 

coefficients form an upper triangular matrix. 

The last of this set of equations involves x„ only, the last but one 
involves and only, the last but two involves and x„_, 

only, and so on. This whole system of equations can therefore be solved 
by starting from the last and working backwards so as to determine ic 
values of in this order, using at each stage the values 

of the x^s previously obtained. This process is known as back sub¬ 
stitution'. In it, the value of each unknown is determined from that 


(8.13) 
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equation which was used as pivotal equation to eliminate it in the 
elimination process. 

In the calculation the usual symptom of an ill-conditioned set of equa¬ 
tions is that the coefficient of x n in the last equation of the set (8.15) is 
small compared with the-values of the coefficients in the equation used 
for the elimination of this is illustrated in the following example. 
Another symptom (of which this is a particular case) which may 
appear is that the elimination of one of the unknowns between two 
equations, say equations (.4) and (5), by forming the linear combina¬ 
tion (A)-rci(B) (with [a| ^ 1), results in an equation in which all the 
coefficients have values which are small compared with their values in 
equation (.4). 

Example : 

To solve the equations 



— 23jTj -f- lla-j-f x 3 

, = o, 

1 lXj - 3Xn 

— °r — 

3, a-! — 

2*2 +*3 = -2. 


Line no. 

Coefficients of 

6 

Check 

sum 

Notes 

operation x x 



(1) 

-23 

11 

1 

0 

-11 

• 

(2) 

11 

-3 

_ O 

3 

9 


(3) 

1 

_ o 

1 

O 

— •» 

o 

A# 


(4) 

= «,xo) -n 

5*261 

O 478 

0 

-5*201 

a, = 1 1/23 = 0*47826 

(5) 

- (2)+ (4) 

2 261 

- 1*522 

3 

3-739 








cross sum 

(6) 

= •*, X ( I ) - 1 

0-478 

0*043 

0 

-0*478 

a, = 1/23 ^ 0-04348 

(') 

- (3H-(C) 

- 1*522 

1*043 

_•> 

-2*478 







-2-479 

cross sum 

(«) 

= aj :< (5) 

1*522 

- 1*025 

2*020 

2*517 

1*522 

a, - - 0*6732 







* 2-261 

(U) 

= (T)*f (8) 


0 018 

0-020 

0-038 







0-038 

cross sum 

(10) 

= 


1 

Mil 

2111 


(HI 

1-52 2s, 


1-522 

1-691 

3-213 


(121 

- (5)-MU) 

2*261 


4-GO 1 

0-052 


(H) 

= s« 

1 


2*075 

3*075 

cross sum checks 

(14) 

- 11 x (13) 

- 11 


- 22*825 

- 33*825 


(15) 

= (1)4-(14) 







-(10) -23 



— 23*036 

-40 936 


( 10) 

- Tj 1 



1*041 

2*041 



1 in,i! check 





IlesiJuuls 


>, - 1041 

— 

23x, -f 1I/.4 

x 3 — —41007 

- 0*007 


j t 2 (iT5 


1 - 3x, 

2j , 3-«04 

-f 0*004 


, Mil 



J, - 1 

•998 

4-0 002 
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Notes: (i) The 1 pivots’ are distinguished in this example by being printed m heavy 
type (in manuscript an underline or ‘box’ could be used), the coefficient of the 
last remaining variable being counted as a ‘pivot’ for this purpose. The largest 
coefficient is chosen as the first pivot, and the result of eliminating ar, is given y 
lines (5) and (7). In these equations the coefficient of r 2 in line (5) is the greatest, 
and this is taken os the next pivot. 

(ii) In line (7) there is a difference of a unit in the last figure between the num er 
derived from the other relevant entries in the same column and that derived from 
the cross sum along the ro«e. This is an effect of rounding errors, and occasional 
small discrepancies in the check, such as this, mu^t be expect* . n le 
calculation, the value derived from the cross sum should ho used m such cases. 

(iii) If the coefficients are known to have exact integral values, then it is signi • 
cant to keep any number of figures in the calculation. In tins example three 
decimals have been kept. Any required accuracy could be attained by keepm 
enough figures, though before the solution is carried out it may be difficult to judg 
how many are needed to give a specified accuracy in the solution. In the pies r t 
case it might be expected that the three decimals kept should bo enough tog 
the solution to 1 per cent,, allowing for rounding errors; but ns will be seen (ui d 

(iv) below), the 'solution' in this case is not even accurate to two figures. 

If thococfficients were only known to two decimals there would be no s.g. , cimeo 
in keeping more than three. If, for example, the coefficient o f were only known 
to lie in the range -3 004-0 01, then the entry 2-261 in line (o) might stand for 
any number incite range 2*20. to 2-271, and the only purpcie of keeping mien the 

third decimal is to avoid rounding errors accumulating in the sec 

(iv) By taking another decimal in 4, only (4, = 10 0,) these re iduids cmtl e 

reduced to -0 001, +0-001, -1-0-002 to three decimals, so that at first s gl t 

would appear that this solution is correct t. at hast two d,--«• 

however: the correct solution is aq — 1. 3 z • » ’ i /.hnrnrt^r 

even correct to one decimal. This is a consequence of the ill-cond tioned 

of the equations. There is a warning of this character of he e,,ua.,ons a lie 9) 

the working, at which the coefficient of 4, is very small compared with that 

q <v) If the coefficients are known to have exact integral values, the'' “' f jj" 

last pivotal coefficient can sometimes be improved ns follow-s. 1 lu | 
piyots is the deter,ni.io.it of the coefficients of the original equations f a d U s 
must bo an integer if the coefficients are integers. In this example the p.oduc, 
the pivots as evaluated is 

— 23 x 2-261 x 0 01S — —0-94. 

The second factor is certainly not in error by more than 1 1 .' U /,''I' iV/ 1 ^,!i'. • 

possibilities of rounding errors cannot affect the tl»n< )\ IIIU,< 1,1 i ...mteiznl 
so the value of this product lies between -0-84 and -104. Hut it must hi 

and must therefore be - 1; hence the value of the lust \n\oi is 

-]/(- 23x2-201) - 0-0192, 

the fourth decimal being certainly correct. ■ , ,i. 

This argument must no, be used unless the coefficients are non;, to. k. 

otherwise it might give a false idea of tlio a<*cmar> <>f 1 " 

t A factor (- 1) .nay he in,re,laced if U.» order of I ho variables or of ,1.0 e,,,i,,„e, 

changed in the elimination j>roco-». 

63i3 M 


is is 
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(vi) If £ lt £ 2 , £ 3 is an approximate solution, and R lt i? 2 , i? 3 the residuals obtained 
by putting Xj = then the corrections (aj—£,) to the approximate solution can 
be obtained by solving the equations (8.3) in a similar manner. 

8.23. Alternative arrangement of the elimination process 

In the above arrangement of the elimination process, each pivotal 
equation is left unchanged in later stages of the process. An alternative 
procedure is to use each pivotal equation to eliminate an unknown from 
the pivotal equations previously used, as well as from later equations. 

In the above example, for instance, the pivotal equation (5) can be 
used to eliminate .r 2 from equation (1) as well as from equation (3). 

This procedure avoids the process of back-substitution, but the 
elimination process is longer, and the total amount of work involved is 
about the same. 


8.3. Inverse of a matrix by elimination 

One way of inverting a matrix A is to obtain a set of solutions of the 
equations ^ 

= (8.16) 

i 

for different sets of values of the b's, namely the sets 

b l = \, b, = 0, b 3 = 0, ...; 

b i = 0, b 2 = 1, 6 3 =0, ...; 

and in general for the sets 

b k =\, b t = 0 (/ # k). (8.17) 

I he solution Xj of the equations (8.10) will depend on the value of k 
in (8.1 1 ). lor any particular value of k. let us write the solution 


= <> 


I hen lor any set of values of the b's in (8.10) 

ik^k* 


=,- V c 
T: 


that. is. the r jk .s are the elements of the matrix which is the inverse of A. 

'I lie solution of the e«,nations for the different sets of values (8.17) of 

the h s ran be carried out simultaneously by a slight extension of the 

juojcs.n * a separate column for the coefficients of each 

value of b k in the original equations. 

h.uiinj)/'': I u .solve I he e<juat ii >ns 

-2^-f 1 I x 2 -f .r.j = 

- 3. 2j 3 = t>„ 


1 1 




2.r. + x 




i ” r ! ; ,!li ’ ' ° n V 1 "" 1 V Thr 1.-ft-hand sides of those equations are the 

' 1 ' | \ |‘>un 1 xiimpl.-; the right hand sides liave general values. In this 
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example tho elimination will be carried out by the special process mentioned in 
§8.21 as avoiding divisions, and the rounding errors associated with them, in the 

elimination process. 


% 

Equation no. 
and operation 

Coefficients 

X, X 2 

of 

*1 

Coefficients of 

6, 6, 6, 

Check 

sum 

Notes 

(ii 

(2) 

(3) 


-23 

11 

1 

11 

-3 
_0 

1 

-2 

1 

1 

0 

0 

0 

1 

0 

0 

0 

1 

- 10 

7 

1 


II II 
S' 

(1) -(3) 

(2) + 2 x (3) 

-24 

13 

13 

-7 

0 

0 

1 

0 

0 

1 

-1 

2 

- 11 

9 

cross sum chucks 
cross sum checks 

(6) = 
(7) = 

(4) + 2x(5) 

(4)+ 12x (0) = x. 

2 

0 

- 1 

1 

0 

0 

1 

13 

•> 

24 

3 

35 

7 

73 

cross sum checks 
cross sum checks 

(8) = 
(9) = 

(G) + (7) 

4(8) = x, 

0 

l 

0 

0 

0 

0 

14 

7 

26 

13 

38 

19 

80 

40 

cross sum checks 

(10) = 
(11) = 

2 x (7) + (3) 

(10)-(9) = x. 

1 

0 

0 

0 

1 

1 

26 

19 

48 

35 

71 

52 

147 

107 

cross sum checks 


and 




24 

3.» 



Henco = 76^136,+ 19*3. 

x z ■= 136,+ 246 a + 356 3 , 

= 196,4-3562+5263, 

II \\ / 7 13 

-3 — 2 

-2 1 

Notes-, (i) In this example advantage has been taken of the simple numerical 
values of the coefficients to lighten the numerical work of t he clnnmatton process. 
The particularly simple values of the coefficients of w, suggest that tins >a the 

unknown to eliminate first. , . c 

(ii) In line «>) no elimination is carried out. but a hneur combination of the 

equations is made so as to keep down the magnitudes of the numbers occumng 

in the calculation. . _ , 

(iii) By avoiding division and so keeping .work free from rounding errors, 

tho exact solution is obtained without any attention having to bo g,von to the 
number of figures kept at the various stages of the work. Lurther the, lleond, tinned 
nature of the equations (see note <iv) below) gives no difficulty , 1 ., obtaining . 
solution. Also the numbers occurring are simple enough ... tins case for the whole 

calculation to be done without the aid of a desk machine. 

(iv) The large values of the elements of the inverse matrix show why at eh a poor 

approximation to the solution, as represented by the 'solution obtained 
givea Huch small residuals. 

IHf approxi.na.ion .0 the solution, and It,, t,. It, are the residuals 
obtained on substituting x, f„ w, {,. w, f, into .be equations, then tlu 
corrections to the approximate solution are 

(x.-f,) -- 7/f,-H3/f s + U»^ s . 

= 13//, I 24//, + 35/f 3 , 

(*»-&) = 19//,d 35/f l +52//„ 
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so that if R l — R 2 = R 3 = 0-01, then x 3 —= 1-06; that is, the error in an 
approximate value of x 3 may be over 100 times the residuals in the equations, 
although in the equations this unknown only occurs with coefficients 1 and 2. 


8.4. Choleski’s method! 

In the elimination method explained above the number of quantities 
written down in the process of solution of a system of n equations is of 
order n 3 . When n is more than about 6 or 8, there may be advantages 
in arranging the work so that the number of intermediate quantities 
recorded is considerably smaller, of order n 2 rather than of order n 3 . 

The method usually ascribed to Choleski, of which there are several 
variants, is one in which the amount of recording of intermediate results 
is reduced in this way. It depends on the resolution of the matrix A of 
the coefficients into the product LU of two matrices, of which L is lower 
triangular with the diagonal elements unity, and U is upper triangular. 
Then the system of equations 


can be written 


Ax = LUx = b 
Ly = b, 


Ux = y. 

Written out, equations (8.18) are 

Vi = K ] 

hl!h~T Vi = I 

^31 #1 + ^322/2 + 2/3 = ^3 j 
• • • • • • • / 


(8.18) 

(8.19) 


( 8 . 20 ) 


since all the diagonal elements of L are unity; from these equations 

Uv - can be obtained in succession. And equations (8.19) written 

out, are 

M n- r i-Hq>^VMq 3 *3-f ... + Win* N = y t ' 

u s 2 X 1 + w 23 *3 + • • • + U 2n x n = y 2 


— } Jn-l 


( 8 . 21 ) 


U nn^n Ifn 

and, once the values of the y/s have been found from (8.20), those of 
the x k '& can be found in succession from equations (8.21), starting with 
the last and working backwards. 

To resolve the matrix A into the product LU the rows of A are taken 

S°o. tV,r oxampk L. 1-ox. H. D. Hu.skoy. and J. H. Wilkinson. Quart. J. Mcch. 

amt 1 (104S). 149; A. M. Turing. ih,U 287. For unotl.or method ufasimilar 

t>pu M*e 1 . I>. trout, / raius. Amcr . I?ust. Elect. Eng. 60 (10 U), 1235. 
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in order, the elements of L being derived from elements of A below the 
diagonal, and those of U from those of A on and above the diagonal. Let 


a 

a 

a 


11 


21 

31 


fl 12 a i3 
®22 a 23 


a l4 

rt 24 


a 32 fl 83 fl 34 


1 


_I ‘21 


31 


0 

1 


0 0 
0 0 


h* 1 0 



U 


V 


12 


22 


0 


V 

u 


13 


23 


V 


33 


U 


14 


Uo 


24 


U 


34 


Then for the first row of A we have 


For the second row of A 

a 21 = ^21 W H» 
a 2r = ^ 2 i w lr+ w 2 r 

For the third row of A 


lr — °\r- 

(S.22) 

i = °2l/ W ll |. 

(8.23) 

V 2r = a 2r —?2l«lr ^ 



= a 


3l/ W ll 


a 31 — ^31 W ll» i . 0 

«32 = + hz = ("s*”'* “»>/"« '' 

a 3r = ? 31 W lr+ ? 32 W 2 r+ W 3 r> V 3r = « 3 r—^"lr" l 32 H 2r (/ 

and in general, for the ;dh row of A 

= («„- 2 (r<p)1 

(r > />) 


(8.25) 


M jtr S 

If the elements of L and U are calculated in the order indicated by the 

sequence of these formulae, then the cv.luat.on «t each- > ' 
unknown, one element of A which has no,, been used - ; 
quantities already calculated. The accun.ulat.on of the pro1; ' 
these formulae, and the divisions by «„ in the calcula turn of ter Imn ts 

of L, can all be done without writing down any .ntermed.ate result 
the quantities which have to be written down m he course of ,1ns 
resolution of A into factors are the »* components of L and U. 
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The work can be checked by t lie use of column and row sums. 
Foxf recommends the following arrangement of the work 


Row sum 



«u 

Cf j 2 

"13 

a u • 

V* 

-“1 

A 

o 2l 

a 22 

23 

21 • • 

V 

—2 


«3I 

• 

«32 

• • 

«33 

• • 

c/ 34 

S 3 


1 

0 

0 

o . . 


L 

Ul 

l 

0 

o . . 



1=1 

• 

• • 

1 

• • 

o . . 


u = 


0 

0 

0 . . 


transpose 

“12 

M 22 

0 

0 . . 


of U 

“l3 

«23 

"33 

0 . . 


s = column sum of U 

• 

• • 

• • 

0 0 


*1 

s 2 

*3 • 

• • • 



Check: Sum [(rth row of L) x row s] ^ row sum Z r . 


In this arrangement, in which the matrix U is written out in the form of 
its transpose, the factors which have to he multiplied in forming the 
sums in formulae occur in the same column, and correct selection 

of them is easier than it would be if one set of factors were in a row and 
the other set in a column. 

The advantage of this method is that the number of quantities which 
has to be written down is only of order m®. On the other hand, too little 
is written down to indicate how the numbers which are written down 
were obtained, and this makes diagnosis of mistakes difficult; the use 
of row and column sums gives a good indication of freedom from mistakes 

when nono havo 1,00,1 »mdc. but the checks arc too few to be of much 
help in identifying or locating a mistake if one is indicated. 

The absence of intermediate results to make it clear how each number 
m the course of the calculation is obtained means that the computor 
must depend on his memory for the required sequence of operations. 

I Ins sequence is slightly different for every one of the elements of L 
and ot l . and though it might become familiar enough to anyone who 
bad much work of this kind to do, the writer's experience is'that it is 
too complicated to he satisfactory for occasional use. The method seems 
to be one lor the professional expert and the specialist rather than for 
the occasional user; but it is so strongly recommended by those who have 
had enough practice to become completely familiar with the sequence 
ot operations required that it has been included here. 

+ *' ox ’ ■ h, "rn. R<»j. Stut. Soc., Srr. B, 12 ( 19 "» 0 ). 120 . 
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8.41. Inverse of a matrix by Choleski’s method 

The inverse of a lower triangular matrix L can be found by taking 
the identity LL" 1 = I, or in expanded form, with C for L _1 



0 

1 


l 


32 


0 

0 

1 


0 

0 

0 



12 


i 22 


l 13 


33 



and working through line by line of the unit matrix. The first line gives 


11 


= 1, 


i; 


= 0 (j>l). 


The second line gives 


= 0, 

21 ^12 “^^22 = 1 ’ 
/ 2 1 C 1J + C 2i = 


Ul C 11 + C 21 


/ 


whence c. n = —/ 21 , 
whence c.,., = 1 > • 
whence f 2 > = 0 > -)» 

and so on. The numerical calculation can be arranged in the same way 
as for the factorization of A into a product LU. The inverse of an 
upper triangular matrix can be treated in the same way, starting with 

the last row of the unit matrix. 

If A = LU, 

then A 1 = U *L 

so that a matrix can be inverted by factorizing it as a product LU, 
inverting U and L separately, and finally multiplying them to form 

A-'. Like the C'holeski method for the solution of a single set of equations, 

however, it is a method which is more suitable for the specialist who has 
a lot of work of this particular kind to do than for the occasional user. 


8.5. Relaxation method 

An indirect method which is very powerful in some eases is one 
called the ‘relaxation’ method. It was originally de\clopul by Soul i 
wellf for application in problems of structural engineei ing, am MJ,m ° 
the terminology of the method is derived from this particular application. 

But its range of application is much wider. 

t R. V. Southwell, Proc. Ho,,. Soc. A. 151 (IlKW). ™: ><“ a *'° 184 (m5,> : 

Relaxation Method* in JCnyirueriny Science (OxfonJ, 
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For any approximation x } = ^ to the solution of equations (8.1), the 
‘residuals’ of the equations are defined by (8.2). For the solution of the 
equations, all the residuals are 0. In the relaxation method attention 
is concentrated on the residuals, and the method consists in making 
changes in the a.-’s in a systematic manner so as to reduce the magnitudes 
of the residuals to negligible amounts. 

The first process is to draw up an ‘operations table’ giving the change 
of each residual for a unit change of each single Xj. Then a set of initial 
values of Xj is taken and the residuals calculated, and changes of the 
Xj s then made in such a way as to decrease the residuals; the steps of 
this part of the work are recorded in a ‘relaxation table’ in which the 
changes of the x's and the resulting total residuals are recorded. 


Example : To find, correct to two decimal places, the solution of the equations 

9x,-2x 3 -fx 3 = 50 

*1 + 5*2-3*3 = 18 . (8.26) 

- 2x, + 2x, + 7x 3 = 19 

For the first equation the residual Ii 1 , for any trial values of x x , x 2 , x 3 is 

i?, = 9x, — 2.r 2 -f x 3 — 50. 

Hence for a change Ax, = 1 of x, alone, the change of if, is Aif, = 9; similarly 
for a change Ax, = 1 of x, alone, the change of if, is Aif, = — 2, and for a change 
Ax 3 = 1 of .r 3 alone, the change of if, is Aif, = +1. These values of Aif, are 
entered in the first column of the operations table. 

Similarly, the residual for the second equation is 

= *i+5* 2 -3x 3 -I8, 

and for the same changes Ax„ Ax,, Ax 3 . the changes in if 2 are Aif 2 = +1, +5, and 
- 3 respectively, and so on. Thus in this arrangement of the work, the matrix of 
these entries in the operations table is the transpose of the matrix of the coeffi¬ 
cients in the equation. The working is shown on the opposite page. 

Xote *\ {i) lu this oxnrn P ,e - « <+«nge 4*i of x, affects if, mainly, and if 2 , if 3 to 
a smaller extent, and similarly for changes Ax 2 and Ax 3 . A change Ax„ made in 

sn.-h a way as to reduce j/?,| considerably, is called a ‘relaxation* of*,, and the 
relaxation process consists of making a sequence of such relaxations. Thero is 
clearly no point m choosing Ax, so as to make if, exactly zero, since if, will bo 
" , ” y S,, ’’^ f l ucnt "-luxations Ax, and Ax 3 . It will usually bo adequate to take 

sm *T " c ,L,t mim >ors t * l< ‘ relaxations; this greatly lightens the numerical work, 
and mnkrs it p^s.ble to carry out the greater part of it mentally and speedily. 

’ ''■ i r< lt>Xa, . ,on tab, ° begins with any trial set of values of x„ x„ x 3 and the 

' «• -!■"'•• »>g residuals. The simplest first trial set is x, = x, = x 3 = 0, and the 

residua i ien ,ust the negatives of the constant terms in the equations. 

. ' "" n " 1 • Ir r-sidunl /f, IS th<-greatest, and a large part of this can be removed, 

, s,M '* |, cr ;-linrige.s in t he other residuals, by a relaxation Ax, = 5. The resulting ' 

• •'ung. - mull «he residuals are given by multiplying line (1) of the operations table 

! ' '.' ample on p. 169. these changes are shown in brackets; they would 

no; be w.-.tten down in actual working; each would be evaluated mentally and 
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Operations 

table 


A*i 

1 

0 

0 


A* a 

0 

1 

0 


Ax 3 

0 

0 

1 


A i?! A R 2 A7?3 Notes 
9 1 — 2 Line (i) 

— 2 5 2 Line (ii) 

1—3 7 Line(iii) 


*i = ° 


x, = 0 


x, = 0 


i?j R z R 3 

-50 -18 -19 


Relaxation 

table 


( 0 ) 
(0) 

1 

( 0 ) 

w 

*1 = 6 


( 0 ) 

( 0 ) 

5 

( 0 ) 

( 0 ) 

-1 


(0) 

4 

(0) 

( 0 ) 

- 1 

( 0 ) 


(45) (5) (-10) Line(i) 


-5 

-1 

-11 

-2 

-3 

-1 


-13 -29 


x 5 


-25 

0 

1 

4 

-1 


-1 

9 

0 
_ o 



- 1 


- 1 


— 2 Check 


x 10 


10*2=60 10x a = 40 10* 3 = 30 -10 -10 20 


-7 - 19 

-15 1 


-1 


-1 


3 
2 

4 


3 

6 

1 

0 


1 

9 

5 

-2 

-4 

-2 



10*, = 61 10* 2 = 43 10*3 = 32 


610 

5 


430 


X 10 
320 


-50 

-5 

-1 

-3 


0 -20 
5 -30 

-7 -2 

— 2 0 


100*2 = 615 100* 2 = 431 1 00*3 = 324 

X 10 

6150 4310 3240 


-3 -2 

-30 -20 

— 38 0 

-2 4 


0 Check 

0 
8 
0 


6154 


4314 


3240 


0 Check 


add*, to the previous value of the S"n 

multiplied by Ax, = 5, gives 46. which ^lin^ X 

It, (= 6) and only tins ia wntten down b „ „ r , ttin , lown 

table the only entries in the /?; columns arc . columns might bo 

in actual working. Similarly, the bracketed zeros in the , 

omitted, as they have been later in the relaxation In ; • • _ 

After the first relaxation A*, = 3 t he largest -.dual ^ ^ 

tins is removed by a further relaxation A. , • ff< ^ thcso rt . luxn tions A* 3 

reduced to zero by A* 3 = 5. The contribu 10 is i 
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and A.r 2 have been such that |i?j| can be reduced substantially by a further relaxa¬ 
tion Ax i == 1, and the procedure continues until the residuals are so small that they 
cannot be improved without making relaxations of magnitude smaller than unity. 

At this stage it is convenient to avoid decimal points by multiplying the entries 
in the relaxation table by 10. But before doing this it is advisable to check that 
the calculat ion has been carried out correctly so far. This is done by adding up the 
changes Ax t and adding the result to the initial value taken (here zero), and 
similarly for .r 2 , .r 3 . The resulting values of x lt x 2 , x 3 are then substituted in the 
equations and the residuals calculated; they should agree with the last line so far 
obtained in the operations table. If they do not, there is no need to go back and 
look for the mistake; these values of x lt x 2 , x 3 form as good a set of trial values as 
the set (0, 0, 0) actually used—indeed they are probably much better—and can 
be used as the starting values of a further calculation. 

(iii) When a set of small residuals has been obtained and checked, the values of x 

and the residuals can bo multiplied by 10 and the process continued, and this 

operation can be repeated as often as required. In this example it is repeated until 
it gives the result 

1000(x,,x 2 ,x 3 ) = (0Io4, 4314, 3240) 


or, to two decimals, (x„ . T „ x 3 ) = (615, 4-31, 3-24). 

It will be spoil that although the number of operations may be large, each is very 

simple and, except for the checking operations, each involves onlv numbers of two 
dibits, ami oftrn only of one digit. 


8.51. Group relaxations 

'I here is no need to restrict oneself to relaxations of the variables 
singly. In this example a relaxation A.r 2 makes appreciable contributions 
to li x and B 3 , and the process of reducing the residuals would clearly 
be quicker and easier if we could make correlated changes in the variables 
in such a way as to affect only one residual considerably. It is often 
possible by trial to find linear combinations of the variables with this 


-V 3 

-Y, 


A.i 

0 

O 

I 


i 


■ we 

have: 



Ax, 

A.r 3 

AR { 

AR., 

1 

•> 

(i 

- I 

4 

- 1 


23 

t 

- 1 

0 

24 

lv 1 
% 

»<• used to 

reduce 

B 3 w 


AR 3 

16 

1 

1 


■ .. similarly a multiple ot A 2 

r: "' " " sr<l "'.Iikv «... Relaxations which arc multiples of A'., or A', 
>" ‘•■■>ll' , <l ero.,,. relaxations' l.y Southwell. Use of them eoneinonds 
" Mi.ikjiii' a linear transformation of the variables so that in the trans¬ 
value hies the non diagonal coefficients in the equations are small 
" lth OH-llieients. Hut we ,lo not have to carry 

' ml " a " sl, " m '"" ,n f"""ally hy introducing the new variables and 

the epaat inns in terms of them. The process of carrying out 
u n 11 nicr 11 -aI work makes this transformation for us 
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Use of these group relaxations does not, of course, preclude us from 
using relaxations of a single variable if the values of the residuals indicate 
that such a procedure would be appropriate. 


Example 


wr w »^ rv 

Ar, 

Ax a 

Ar 3 

A/?, 

a /? 2 

A/?3 

Operations 

1 

0 

0 

9 

1 

— 2 

table 

0 

1 

0 

-2 

a 

5 

0 

** 


0 

0 

1 

• 

1 

-3 

/ 

x a 

1 

4 

- 1 

0 

24 

- 1 

x. 

0 

1 

2 

0 

-1 

16 

Relaxation 


x 2 



Ii 2 

R 3 

table 

0 

6 

0 

-50 

-18 

- 19 


5 



-5 

- 13 

-29 

2X 3 


2 

4 

-5 

— 15 

3 

x* 

1 

4 

- 1 

— 5 

9 

0 

40 


6 

6 

3 

-5 

9 

2 Check 




y 10 






60 

30 

-50 

90 


— 4X a 

-4 

- 16 

4 

-50 

-6 

24 


5 



— 5 

- 1 

14 

-A'» 


- 1 

-2 

-5 

0 

_ 0 

40 


61 

43 

32 

— 5 

0 

- 2 Check 




X 10 





610 

430 

320 

- 50 

0 

-20 


6 



4 

6 

- 32 

2 X 3 


2 

4 

4 

4 

0 


616 

432 

324 

4 

4 

0 Check 




X 10 





6160 

4320 

3210 

40 

40 

0 

- 2X, 

-2 

-8 

0 

40 

-8 

0 

44 


-4 



4 

- 12 

10 




_2 

•» 

-6 

- 4 



1 


0 

- 1 

_ 2 


6 1 f>4 

4313 

3240 

0 

- 1 

— 2 Check 


6-154 

4-313 

3-240 


Solut ioi 

• 


8.52. Use and limitations of the relaxation method 

Teinplef has shown that if the simultaneous equations are deii\ed 
from a positive definite quadratic form, then the relaxation pioeess 
formally converges. However, this is neither a necessary nor a sullieient 
condition for it to be a practicable process in actual numerical woik, it 

f G. Toinnlo, Proc. /toy. Soc. A, 169 (I93K), 476. 


I 
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is quite impracticable for the ill-conditioned equations (8.6) which can 
be derived from a positive definite quadratic form, whereas it is quite 
practicable for equations ( 8 . 26 ) which are not so derivable. What 
matters much more is that it should be easy to find a set of relaxations, 
either of individual variables or group relaxations,'each of which has a 
considerably greater effect on one of the residuals than on any other. 

Although the relaxation method is one of successive approximation, 
there is no limit in principle to the accuracy to which the solution can 
be taken. In the structural engineering context for which the method 
was originally devised by Southwellf there is no significance in carrying 
the solution beyond a certain limited degree of accuracy, and its first 
presentation in this context appears to have given the impression that 
it is fundamentally approximate in character; but it is no more so than 
any other numerical process which is subject to rounding errors. When 
it is used in contexts in which no approximation is involved in the 
equations to which it is applied, it may be possible, and significant, to 
carry the calculation through to a relatively high accuracy; and because 
of the simplicity of the process this may be the easiest way of deriving 
results to such accuracy. 

For example, the recurrence relation for the Bessel functions 

«4+i(*) —(2»/*) «4(*)+«/n-i(*) = 0 

provides a set of simultaneous linear equations for J„(x) as a function 
of n for given .r; and given, say, Jjx) for n = N ^ x and the condition 
-> 0 {ls n -> oo, it is possible to solve these equations to any accuracy 
required by an application of the relaxation process. The process is a 
simple and quick one, and Foxj has recorded that starting only from 
J 10 ( 10 ), he has obtained eighteen-decimal values of J„(10), for values of 
n from 11 to the value (n = 37) at which J„(10) < 10~ 18 , without 
difficulty. This process is most effective for calculating J„(.r) for n > x , 
which is just the range over which the use of the recurrence relation to 
evaluate «/„ fl (x) from J v [x) and «/„_,(rr) becomes unsatisfactory (see 
§11.3). 

In one particular application, namely to the solution of ordinary 
differential equations with two-point boundary conditions (see § 8.53) or 
of partial differential equations of elliptic type (§10.2), the relaxation 
process is combined with the use of finite-difference approximations to 
derivatives, and this is perhaps part of the reason why the relaxation 
process has come to be regarded as essentially approximate. But this 

t R. V. Southwell. Pror. Roi/. Sor. A. 151 (1933), 50. 

+ L. Fox, Proc. Camb. Phil. Soc. 45 (1949), 50. 
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is a mistaken idea; the approximation is in the reduction of the differen¬ 
tial equation to a set of simultaneous algebraic equations by use of finite 
differences. This approximation is involved whatever numerical process 
is used for the evaluation of a solution of these simultaneous equations, 
and the relaxation process, if used, is quite distinct from this approxima¬ 
tion. 

If a number of sets of equations with the same set of coefficients but 
with different values of (b v ... } b n ) have to be solved, no advantage can be 
taken of this in the relaxation process, and for this reason this process 

is not well adapted to the inversion of a matrix. 

The main difficulty likely to arise in using the process is slow conver¬ 
gence, and this will be most likely to occur when many of the coefficients 
in the equations are of the same order of magnitude, so that a relaxation 
of one of the unknowns makes changes of similar magnitude in a number 
of residuals. This situation is likely to occur when the equations are 
ill-conditioned, but is not necessarily a symptom of this condition. 


8,53. Linear differential equations and linear simultaneous 
equations 

Consider the linear differential equation 

y" = f( X )y + ,j(x) ( 8 - 27 ) 

with two-point boundary conditions, y = y 0 at x = •'«’ } J ~ 'Jn at 
x = x = x 0 + 7 i(hx). To the approximation in which (8.r ) z y can be 

replaced by 5 hj, this differential equation is equivalent to the set 
of algebraic equations 

8 ~yj = 

or { *~ H) 

for 1 < j < » -1, with y. and y„ given. This is a set of linear algebra* 

equations for (»- 1) unknowns, and could be solved numerically by any 

of the methods for the solution of such equations. 

A convenient process in many cases is one that has been suggested 
and developed by Thomas and by Fox. It is a version of the Cholesk, 
method <§ 8.4), which in this case is very simple because of the spcciall.N 

simple forms of the matrices concerned. 

Equations (8.28), arranged with the known quantities on the right- 
hand side, are 

-[2+(S*)7 1 ]y I +i'it = -»«( 
y/-i-[2-H 8 *)7i]j'>+*'m - (2 <3 < 


(8.28 a) 
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For short, let <f> j = (8x)% and 

C i = -**+&*)%. cj = (**)% (2 < j < n-2), c n _j = -y n +$x) 2 g n - x \ 

also let y = (y x ,y 2 ,...,y n ), c = (c l ,c 2 ,...,c n ). Then equations (8.28a), 
in matrix form, are 

where A is the matrix 


Ay = c, 


(8.28 b) 


A — ( — {2+<f>y) 1 0 0 

1 — ( 2 +<&») 1 0 
0 1 ~(2 + £,) 1 


0 

0 


0 

0 


0 

0 


0 

0 


0 

0 

0 


0 

0 

0 


1 -(2+^) 
0 1 


0 

0 

0 

1 


~( 2 +<f>n-i)J 

It can easily be verified that for this matrix A the lower and upper 
triangular matrices of § 8.4 are 


1 

0 

0 . . 

0 

0 

0\ 

-h 

1 

0 . . 

0 

0 

0 

0 

• • 

-k 

• 

1 . . 

0 

0 

0 

0 

• 

0 • 

• • • 

0 . . 

• • 

-In-, 

• • • 

1 

0 

0 

0 

0 . . 

0 

^»i — 9 

1/ 


(8.28cJ 


U= /_ 


l/'i 

0 

0 


1 

-iU, 

0 


0 

1 

-1/'3 


0 

0 

0 


0 

0 

0 


1 


I. ^,1-2 

0 - 1/W 


<> o 0 

0 o 0 

Where I//, = (* + *). = (2+^) (2 <i<n Li)‘ 

ioth.u /, = I;( 2 + M Ij = 1,(2+^—(2 < j sj „_I). (8.28d) 

of“: g , r t,ons ,he ,/s can be evaiuated in — 

Also equation (8.28 b) becomes LUy = c, so that if z = _Uv, then 
Z c, and for the matrix L given by (8.28 c) it follows that 

. = ~ C i’ z ‘ = ~ e l+ , i-i s i-i (2<i<n-l), (8.28e) 

from which the s/s can be evaluated in succession, in the order of j 
increasing, then from the equation z = -Uy it follows that 

Un~\ Z„- |, = /,(y /+1 + Zj) 2) (8.28f) 

decreasing "'e ^ ^ evaIuated in succession, in the order of j 
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' The process is a direct one, and, if solutions of (8.28 b) are required 
for the same function f(x) and several different vectors c, the matrix A, 
and so the values of the 1/s, are the same for all vectors c, so the evalua¬ 
tion of the i/s only has to be done once. This may be necessary if it is 
required to improve the approximation to the solution of the differential 
equation (8.27) by including higher orders of differences in the rep ace- 
ment of derivatives by differences. The process is easily adaptable to 
second-order linear differential equations containing a first-derivative 
term, and was indeed formulated by Thomas and by Fox for tins more 
general case, but is particularly simple for equations like (8.27) for which 
the first derivative is absent. It can also be adapted to deal with other 
forms of two-point boundary conditions. Also higher-order differences 
can be included in the replacement of y" by finite differences, as follows. 
The set of equations (8.28) has been obtained by replacing the denva 

tive in (8.27) by the simplest finite-difference approximation to ‘t. In 

the solution of the equations in the approximate form (8.-8), y » never 
evaluated; so in taking higher orders of differences into account, it 
more convenient to use those of y than those ofThe next approxima¬ 
tion to y” is given by (Sx)2y; = 8Vi _, s ^ ; 

use of this approximation in (8.27) gives 

+l —[2 + (SurIVjl.Vi + */j (829) 

This set of equations should be solved by an iterative process, the values 
of h'y on the right-hand side in one iteration being obtained from 
results of the previous iteration; higher orders of differences of y can be 
included on the right-hand side of (8.2-)) if appreciable. One should 
r 0( try to solve it by expressing in terms of function values and 
treating (8.29) as a recurrence relation between five succrssivt \a i t 
of , this would be equivalent to replacing equation ,8.27, by a fourth- 
order equation and would probably introduce spurious solu >ons 
The terms tf'y, (and higher diflerence terms if appreciable) can 

incorporated in the c i 8. 

Example: To find an approximation to the solution of the equation 

y" = xhj-\ < 8 - 30) 

Wi the equation and *e V 

y is an oven function of*. Hence we need only consider the rang 

impose a condition of symmetry about x - 0. _ , quatiun (8.28) 

Let us take hx = J and reckon j from j = 0 at a. - U. I 

becomes . , , o, r.. r i 7, (8.31) 
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and the condition of symmetry, y_ t = y lt gives for j — 0 the equation 

— 2/o + !/i = — i + (8.32) 

so that 

^j=*y*.*o= lj«o= -i+*S 4 yo.<V= - *+**!& (1 <j < 7); (8.32a) 

and equations (8.28 d), (8.28 e) apply for 1 < j < 7. 

The working can be arranged as shown on p. 177; some intermediate results, 
which it would not be necessary to write down if working with a desk machine, are 
included to show the sequence of operations. 

In this working, the calculation is split into three sections by heavy lines. The 
first section, to the left of the first heavy line, is concerned with the calculation of 
the 1/8 . The values of (2 + <f>j) are filled in first from the formula for <f>j, then the 
Z/s calculated in succession from formulae (8.28d). The second section is con¬ 
cerned with a first approximation to y jt neglecting the fourth-difference terras in 
(8.31), (8.32). The z/s are calculated first, by working downwards through the 
columns to the left of the thin line; each value of z i is multiplied by the value of lj 
in the same line, and the product is written in the next lower line in the column 
headed Zy_jand added to the ( — Cj) in that line to give the next Zj. 

The next three columns are concerned with the evaluation of yj starting from 
»/ H = 0 and working upwards ; as each yj is calculated it is entered in the next higher 
line in the column headed y j+l and added to the in that line; the sum is then 
multiplied by l, in that line. 

The next three columns are concerned with the evaluation of the 8*y terms in 
formula (8.32 a). The third section of the calculation is a repetition of the procedure 
of the second section with the 8‘y, contributions to the c/s included. 

Notes: (i) The symmetry of the solution about y = 0 has been used in evaluation 
8 s y 0 and S'y 0 . 

(ii) Since y is required to be zero at x = 4 (j = 8), */"(4) = — 1, so 

8 2 i/ 8 = (Sx)*y''(4) + 0(8x) 4 = —0-250 approximately. 

This value (enclosed in brackets) has been used to give a value of 8*y-. 

(iii) The calculation could be repeated with values of 8*y derived from the second 
iteration. 

(iv) .Smaller intervals (8-c) should be used if greater accuracy in the results were 
required. 

(v) This process may not be satisfactory when the function/ in equation (8.27) 
is negative over a considerable range of x. 


An alternative procedure is to use a relaxation process.f It is best to 
•start such a process with quite a coarse subdivision of the range over 
which the solution is wanted (such as 8r — I in the above example) and 
to divide it further as the approximation improves. Bevond a certain 
stage of the subdivision, depending on the behaviour of y as a function 
of c and the accuracy required in the solution, it is often better, as 
suggested by Fo.x.f to keep some higher differences in the replacement 
of //" by linite differences. 


1 L. l*ox. I'ror. Cumh. Phil. Hoc. 45 (1949), 50. 
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8.6. Characteristic values and vectors of a matrix 

The ‘characteristic values’ (also called ‘latent roots’) A of a matrix 
are those numbers for which the system of equations 

Ax = Ax (8.33) 

has a solution other than x = 0, and the solution x in such a case is called 
a ‘characteristic vector’ (or ‘latent vector’) of the matrix or a ‘charac¬ 
teristic solution’ of the equations. 

Written out, the system of equations (8.33) is 

( a ll _ ^*)*l + ®l* a? S+®13*S+— — 
a 2 l^l+(a 2 2 —^A)*2 + 0 23- r 3+— = 
a31 X l + a 32 X 2+i a 33 — A)*3 + — = 

and the condition that this set of equations should have a non-trivial 
solution is that the determinant of its coefficients should be zero. An 
algebraic equation of the nth degree in A can be obtained by multiplying 
out the determinant, though for n > 2 it is seldom that this provides 
the easiest or quickest way of evaluating characteristic values numeri¬ 
cally. But this formal equation gives three results which are useful in 
the numerical work. The equation is 



(— 1)"[A" — a, JA- 1 + ...J+D = 0, (8.35) 

where D is the determinant of the matrix. Hence 

(i) the equation for Alias n roots (multiple roots, if any, being counted 
according to their multiplicity); 

(ii) the sum of the roots is the sum of the diagonal elements of the 
matrix; 

(iii) the product of the roots is the determinant of the matrix. 

The characteristic values A, arranged in order of decreasing j A|, will be 
written A fl> , A (2) , A' 31 ,..., and the solution x } for A = A 1 ' 0 will be written xj p) . 
Since equations (8.34) are homogeneous, each solution x ,p> is undeter¬ 


mined to the extent of an arbitrary multiplying factor. In the formal 

treatment of these equations it is convenient to regard this multiplying 

constant as chosen so that 2 ( x j V> ) 2 — 1; such a solution is called 

J 


normalized'. 


But for numerical work it is usually more convenient to 


take it so that the greatest in magnitude of the .r.-’s is unity. 

J V- 

A commonly occurring type of matrix is a symmetrical matrix, and 
symmetrical matrices have several properties which are useful in 


numerical calculations. The characteristic values A are all real, and the 


} 
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characteristic vectors for different values of A are orthogonal, that is to 

say, if AW # A<«\ then y =0- (8.3C) 

m * J J 
J 

and if any value of A is an mi -fold root of the equation (8.35), then m 
mutually orthogonal solutions x } can he found for this value ot A. Hence 
it is no restriction to take all the characteristic vectors as mutually 
orthogonal, and it will be assumed that they are so taken. 

Further, the behaviour of the quantity 


A = 


(| «;***)/( 


(8.37) 


as a function of the vector x lias a property which makes it useful m 
the approximate calculation of characteristic values. In matrix notation 

it can be written 


A = (xAx)'(xIx), 


(8.38) 


where I is the unit matrix. 

Consider first the result of evaluating A for one of the characteristic 
vectors of the matrix, say for x = x<">. From the definition of a charac¬ 
teristic vector, it follows that Ax'"' = A<"> x<"\ and hence that the value 
of A obtained is just A = A (p \ the corresponding characteristic value. 
Now consider the value of A evaluated for a vector x differing slightly 
from xW, say for x = X<*>+5. From (8.38), its difference from the value 

A (p) is given by (A _A ( "')(xlx) = x(A— A (p) I)x. ( 8 - 3<J ) 

Since x <p) is a characteristic vector with characteristic value A'"' it. 
follows that (A-A ( ">I)xW = 0, and hence 

(A-AWI)x = (A-AWI)?. (8 - 40) 

' Since A is symmetrical, so is A-A^I; hence from (8.39) and (8.40) 

(A— AW)(xIx) = x(A-AWI)^ 

_ ^(A_A (p) I)x (since A—A' p) I is symmetrical) 

_ j: (A—A (p) I)^ (by a second use of (8.40)). 

Hence A differs from A‘ p > by a quantity which is second-order m \\ in 
other words, the quantity A defined by (8.37) is stationary for sma 

variations of the vector x from a characteristic vector. 

Hence from a fair approximation to a characteristic vector a relal i vely 
good approximation to a characteristic value can be obtaine >y 
evaluating formula (8.37), and from a good approximation to a charac¬ 
teristic vector a much better approximation to a characteristic \a ue 
can be obtaine<l. In application to the equations of vibrating systems 
of several degrees of freedom, characteristic values represent squares of 
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frequencies of normal modes of vibration; an account of the use of 
formula (8.37) to determine characteristic values in this context, and 
developments of it, is given in Rayleigh's Principle , by Temple and 
Bickley.f 

It is sometimes convenient to express a symmetrical matrix in terms 
of its characteristic vectors and characteristic values. Let xx stand for 
the matrix whose {j ,&)th element is x } x k . Then if the characteristic 
vectors x (y) are normalized, the required expression is 


A = 2 A^>xb»>x^>; 


p 


if, as is often more convenient for numerical work, they are not^nor¬ 
malized, we have _ / , , 

A = 2 [A<*>x<*>x0>y J (8.41) 

The characteristic vectors of the inverse A -1 of a symmetrical matrix 

A are the same as those of A itself, and the characteristic values of A -1 

are the reciprocals of the corresponding characteristic values of A, so 

that r . , 

A_1 = 2 [(1/A<*»)x0>>x<*y]> (*j p) ) 2 j. (8.42) 

Formally, this provides one method of inverting a matrix. But it may 
be more convenient in numerical work to invert the matrix by some 
other process, and then use (8.42) to determine the small characteristic 
values of A and their characteristic vectors. 


8.61. Iterative method for evaluation of characteristic values 
and characteristic vectors of a symmetrical matrix 
The characteristic value A U) of greatest magnitude can be found as 
follows. Take an arbitrary vector x (0) , with components .r (o)> , of which 
the greatest in magnitude is unity. Form Ax (0) and express it as a mul¬ 
tiple A(j) ol a vector x (1) whose component of greatest value is unity. Then 
repeat the process with x (1) in place of x (0) to give a vector x (2) and so on. 
That is, form a sequence of numbers \ m) and of vectors x (m) so that 

Ax (m-1) = \,n) X (m)» 

where each \, n) is chosen so that the component of x { , n) of greatest 
magnitude is unity. Ihen, unless the vector x< 0 ) happens to have been 
taken orthogonal to the characteristic vector x (1> of the matrix, 

\m) -** A (l > and x< m) -> x (1) as >n -> oo. 


The process thus far is also applicable to non-symmetrical matrices. 

1 he rate at which successive values of \ m) ultimately tend to their 


1 G. Temple and \V. G. Bickley, Rayleigh'* Principle (Oxford, 1933;. 
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limit depends on |A<»/A«|, and is greater the greater the value of this 
ratio. If after the first few repetitions of the process of calculating 
^ x Ax, n) , the successive values of \ m) seem to be tending to a 

Hmit only slowly! the reason may be that ,*»/*’>,-1 is small. But it 
may be that X (0) happens to have been taken nearly orthogonal to x , and 
in case it has been so taken, it is as well to start the calculation again 
with another vector x (0) roughly orthogonal to the one previously used. 

Example: To find x‘ l >, A“> for the symmetrical matrix 


A = 


(This is the matrix of the coefficients of tile equations considered in the examples 

of§§ 8.21 and 8.22.) 

Starting with x (0) =(1,0,0) wo have 



x, 0 , = ( 1 , 0 , 0 ), 

X(i) = (1,-0-48,-004), 

X(2) = (1,-0-442,-0-068), 
X( „ = (1, -0-4461, -0-0649), 

X(1) = (1,-0-4458, -0-0653), 


Ax 10 ) = (-23,11,1) 

= -23(1,-0 48,-0 04), 

Ax (ll = (-28-32,12-52,1-92) 

= —28-32(1,—0-442,—0-068), 

Ax,,, = (-27-93, 12-462, 1-814) 

_ —27-93(1,—0-4461,—0-0649), 

Ax,., = (-27-971,12-468,1-827) 

= —27-971(1,—0-4458,—0-0653), 

Ax = ( — 27-9691.12-4680,1-8263) 

_ —27-9691(1,—0-44579,—0-06530). 

Hence, to three decimals in A' 1 ' and five in x' 1 ' 

A-)= -27-969. x"» = (1,-0-44579, -0 06530). (S 43) 

c n \ m\rl x can be kept small in the early stapes 

NoU: The number of figures m A, e end X , can P r etition of 

and increased as the approximation to A . x impro 
iterative process. 

If the matrix is symmetrical, then having found A<*>, X<” we can evaluate 
the matrix and subtract its contribution from A (see formula 

(8.41)), so forming the matrix 

(8.44) 


A_A<»x <1 >x‘ ,, /2 = A<»>, say. 


For this matrix the characteristic value of greatest magnitude is A<=> 
and this, and the corresponding vector x«>. can be found smnlarly. and 

the process repeated. . . f pnn i 

This procedure, however, suffers from progress.ve loss of s.gmhoant 

figures as the successive contributions (- T j ) jre 

traeted oil; it also requires that each x>»> should he determined 
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accurately before calculation of the next is undertaken. A method free 
from these defects is given in §8.63. 


8.62. An alternative process 

An alternative process, after determining A‘ l> and x (1) , is as follows. 
Instead of eliminating x (1) by evaluat ing A (1, x (l) x (1, /^T (aj 1) ) 2 and forming 

' 3 

the matrix A (1) (see (8.44)), the procedure used for determining A (1) and 
x (1) is repeated, with the modification that x (0) and each x (m) is constrained 
to be orthogonal to x (1) before being multiplied by A. That is, we form 
a sequence of numbers A< m) and vectors x (w) by the relation 

'V/n> X (m) = Ax <m- D —P(m) xU> , 


fi, m) being a number determined by the condition that x (m) should be 
orthogonal to x (1) ; A< m) , as before, is determined so that the component 
of x (m) of greatest magnitude is unity. 

If the work could be done exactly without rounding errors, then there 
would be no need to introduce the multipliers y (m) ; if x (0) were taken 
exactly orthogonal to x H) , then each vector Ax, m , would be orthogonal 
to x (,) . The rounded values of Ax,,,,.,, will, however, contain a small 
multiple of x (1 ', and if this is not removed, it will give rise to an error 
which increases with further repetition of the iterative process, so that 
ultimately x (w , would tend to x a> and not to x ,2> . 


\\ hen x a) and x 12 'have been determined, the successive approximations 
to x <3> are similarly constrained to be orthogonal both to x (1) and to 
x <2> , and so on. 

This process has the advantage that there is no loss of significant 
figures as the work proceeds. On the other hand, since it makes use 
of the orthogonal property of the characteristic vectors, this property 
cannot be used as a cheek, whereas it forms a valuable check when the 
characteristic vectors are determined independently of one another. 
Also it is necessary to determine each characteristic vector accurately 
before starting to calculate the next. 


8.03. Richardson’s purification process for characteristic 
vectors 

In determining a characteristic vector x 1 '' 1 and corresponding charac¬ 
teristic value A •” by a method of successive approximation, it is an 
ad\ antage to Mart u ith a trial approximation w hich is nearly orthogonal 
in nn\ characteristic vectors about which something may be known from 
• ailifi stages <>i t lit' calculation. One method of doing this has been 
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outlined in § 8.62. Another process has been suggested by L. F. Richard¬ 
son,t and has two advantages; first, it does not require accurate deter¬ 
mination of other characteristic vectors, even those for which |A (p) | is 
greater than \X lq) \ for the characteristic vector sought, and, secondly 
it enables the characteristic vectors to be calculated independently of 
one another so that the orthogonal property is available as a check. 

Any vector x (of n components) can be expressed as a linear combina¬ 
tion of the characteristic vectors of A: 

x = 6 1 x< 1 >+6 2 x< 2 >+...+&„x‘"> = 2 b p itfp\ (8.45) 

1 V . 

Multiplication by (A— /I) gives 

(A—;i)x = 2 *'”• < 8 - 40) 

P 

If one of the characteristic values, say A<->, were known exactly, 
and l were taken to he A" the coefficient of the corresponding term 
in (8.46) would be zero; that is to say, (A-A«I)X is orthogonal to x >. 
Similarly (A—A (r, I)(A-A<*'I)x is orthogonal to x< rl and to X ’, and so on. 

Even if AP is not known exactly, the coefficient of **» in (8.46) will be 
relatively small provided that the value of l taken, say l ,r \ is such that 
|A<W_?H| is substantially smaller than most of the other quantities 
|AUD-i<n| for p ^ r. Further, by repeating the multiplication by 

(A — /I) we have (A _/j)m x = £ {\W—l) m b p xP\ 

V 

so that starting from any arbitrary vector x, the vectors ( A ~ /,r)I)x * 

(A—i ,r) I) 2 x,... arc more and more nearly orthogonal to x . Similarly if 
Hr) p> are approximations to A">,A<*>, and any vector x is multiplied by 
(A— m){A-l (s) l), the result will be a vector nearly orthogonal to 

x (r) and to x (8) , and so on. 

If x<'i> is a characteristic vector to be determined, an arbitrary \ ector 
X can be described as a mixture of the required vector x<«> with impuri¬ 
ties’ in the form of multiples of the x--»’s (p # g) in amounts represented 
by the coefficients b„ in (8.45). The effect of multiplying by (A-f I) 
or by (A-F«lr can be described as ‘purification’ of the mixture by 
-removal of most of the x"> component from it, and similarly for the 
effect of multiplication by (A—f'«»I)”; this suggested the term putif.ca- 

tion process’ used by Richardson. 

In order that such a purification process should be effect \ c, the 
values used for quantities like («,(■«’ need only be approximations to 
characteristic values. This has two advantages m practical numerical 

+ Phil. Tran*. Ilo'J. •‘'oc. 242 (11>50). 439. 
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work. First, for symmetrical matrices good approximations to charac¬ 
teristic values can be obtained, by use of formula (8.37), from approxima¬ 
tions to characteristic vectors which are only moderate; hence if only 
one characteristic vector is required, it is not necessary to determine 
the others to any great accuracy. And, secondly, it is possible to use 
simple rounded values for the Vs if this would simplify the numerical 
work. 


Example : To determine x (2) , x (3) for the matrix 

( -23 11 1\ 

without accurate determination of x (1> . 

Suppose that in the example of § 8.61 the successive approximation for x (1 \ A u) 
1ms only been taken as far as the second stage, with x (n == (1, — 0-48, — 0-04), and 
it is desired to find the other characteristic vectors and values. The best available 
approximation to A (1) at this stage is given by evaluating formula (8.37) for this 
vector x U) , and is —28 00. This, however, is so close to the actual value of A (1) for 
this matrix that use of it would not illustrate how effective the purification process 
can be although ( (1 > is only a rough approximation to A (,) . So in working the example 
here, the value i ,l > = — 27 will be taken. Then 

/ 4 11 1 \ 

A —f«»>I = A + 27I = II 24 -2 1. 

\ 1 -2 28/ 

This suggests x = (0, 1, 1) as a trial vector. For this vector x, 

(A27I)x = (12,22,26), 

for which the approximation 24(1, I, 1) is adequato at this stage. Hence we start 
the approximation for the second characteristic vector with x (0 , = 4(1,2, 2). This 

giVGS Ax t0) = 4(1, 1,-1). 


This differs considerably from x !0 „ so that although this x (0) is approximately 

orthogonal to x 1 ”, it is far from x<->(we shall sec later that it is much more nearly 

in the direction of x lj) ). It is also some way from being orthogonal to tho best 

approximation available to x (l >, namely (1,— 0-4S,-0-04), so that, rather than 

using (I, 1,-1) as the next trial vector x ;i) , wo repent the purification process, 
which gives 

(A -f 27I)( 1, 1. — I) — (14.37,-20) = 37(0-38,1,-0-78) 
and take x (1) — (0-38, 1, —0-78) as the next trial vector. This gives 

Ax (1 , - (1-4S, 2-74, 2 40) 2-74(0 54,1,-0-88) 

and (x (I1 Ax.jjVlx,,, Ix m ) - 2-05. 

Th-- further procedure depends on the results required. We will first consider the 
improvement of the approximation to x l2 >, and then the determination of x< 3 > 
without more information about x'-Mhnn is expressed by (8.47) and (S.4S). 

(u) Improve■ t of the approximation to x ,2 >. We now have the approximations 

A" - — 28-0, A' 2 90. and the general result that the sum of tho characteristic 


(8.47) 

(8.48) 
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values is the sum of the diagonal elements of the matrix, in this case - 2o. Hence 
we can conclude that A' 3 > is roughly 0 05, so that l < 3 > = 0 is a good approximation 
to and multiplication by A itself will be effective in removing from a trial 
vector x any ‘impurities* in the form of multiples of x< 3 ». Hence starting from 
(0-54,1, -0-88) given by (8.47) as the best approximation yet available to x -, we 
multiply by (A-f 271) to remove multiples of x“> and by A to remove multiples 
of x (3> and to examine the successive approximations of Ax to A'-'x. 

Since i« 3 » = 0 appears likely to be a better approximation to A' > than l -Zi 
is to A« l > it will be best to carry out two or three multiplications by (A + 2/I) for 
each multiplication by A; and since the vector (0-54,1. -0-88) is already the result 
of a multiplication by A, and so contains only a small admixture of x , lot us start 
by somo multiplications by (A + 27I): 

(A + 27l)(0-54,1, —0-88) = (12-28, 31 -70, - 26-10) 

= 31 70(0-387,1, -0-823), 

(A + 27I)(0*39,1, -0-82) = 29-93(0-392.1, -0-821), 

(A + 27l)(0-392, 1, -0-821) = 29-954(0-3922, 1, -0-8211). 

This has produced a vector nearly free from x“> but possibly still containing traces 
of x (3) . To remove these wo multiply by A: 

A(0-3922,1,—0-8211) = 2-9564(0-3918,1, —0-8216). 

Tho results of further multiplication by A alone diverge. This can bo seen as 
follows. Let x be an approximation to x l2 >: 

x = X (2> + &i X <1, + &3 x ,3) , 

whero b x and b 3 are small compared with 1. Then 

Ax _ A ,2 >[x ,2) -f (A ,,, /A‘ 2) )^i x,,) + (A (3, /A ,:, )5 3 x' 3 ’]. 

Now |A» I >/A , *>| is greater than 1, and in this example it is about 10 so that any 
•impurity’ in x in the form of a multiple of x“> is more prominent in Ax than ... x. 
This building up of x<» can be avoided by further multiplications by (A-1 I). 

(A + 27I)(0-3918,1,-0-8216) 29-953(0-39215,1,-0-82172), 

(A + 27I)(0-39215,’l,-0-82172) -= 29-9571(0-39212,1,-0-82171). 

This vector x = (0-39212,1, -0-82171) gives 

Ax = 2-9567(0-39217, 1,-0-82171), 

(xAx)/(xIx) = 2-9568. 

So that to four decimals in A< 2 >and a possible error of one or two in the fifth decimal 

in x«> A<*>= 2 - 9568 , x‘*> = (0-39212, 1,-0-82171). (8.49) 

Notes- (i) Tho convergence would be much quicker if tho value l' 1 ' — — -8 insti ad 
of -27 were taken, but this working illustrates that the purification process is 

effective even if (<» is only a rough approximation to A'”. nlirn .. ri cal 

(ii) With a matrix whose elements arc integral, as in this exa I , 

w«k is Simplified if .ho I's arc taken to have integral value,, m ^ t h , 
elements of A and of (A-f'D are number, of two d.g.ts only, winch .nukes tho 

The most striking dlustratim. of the powered the 
method is provided by the determination of x« >, the character.* mtm . 

smallest value of |A|, without requiring that x (l) or X <2) should >c < 



186 


SIMULTANEOUS LINEAR ALGEBRAIC 


accurately than they are at the stage of the calculation reached at formulae (8.47) 
and (8.48); comparison of formula (8.47) with the result (8.49) shows that the 
approximation to x (2 > at that stage is decidedly rough. 

We have used i f,) = — 27 os an approximation to A (l) , and can now adopt i (2 > =4-3 
as an approximation to A (2> ; so repeated multiplication of an arbitrary vector x by 
(A427I)(A — 31) will produce a sequence of vectors whose directions will converge 
to that of x ,3> . Now 

4 

(A427I)(A—31) = ( 11 


11 

1\ 

/— 26 

11 

24 

- 2 ) 

11 

-6 

-2 

28/ 

\ 1 

-2 - 

-18 

24 

20\ 


24 

19 

33) = 

B (say). 

20 

33 

51/ 



and we have seen that x = $( 1,2, 2) is nearly orthogonal to x ll) and differs consider¬ 
ably from x |2) , so let us take it as a first approximation to x (3 >. Then successive 
multiplications by B give: 

B( £, 1, 1) = -(35,64.94)= - 94(0-37,0-68, 1), 

B(0-37, 0-68, 1) = -(29-66, 54-80, 80-84) 

= - 80-84(0-367,0-678, 1), 

B(0-367,0-668,1) = - 80*711(0-3675,0-6776, 1), 

B(0-3675,0-6776, 1) = - 80-7108(0-36733, 0-67766, 1), 

B(0-36733, 0-67766, 1) = - 80-7094(0-36739, 0-67763, 1), 

B(0-36739, 0 67763, 1) = —80-7096(0 36737,0-67764, 1), 

B(0-36737, 0-67764,1) = - 80-7095(0-36738, 0-67764. 1). 

This process could be continued indefinitely, to give as many decimals in x (31 as 
might be required. With a possible error of 1 in the fifth decimal 

x ,3 » = (O-30738.0-67764, 1) 

and with this value of x (3 > 

(x' 3 >Ax< 3 >);(x< 3 >Ix 3 >) = 0-01209;,, 

which is therefore an approximation to A ;3) . 

The collected results for this matrix A are 

A' 1 * = -27-969. x'" = (1.-0-44579,-0-06530' 

A <2) = 2-9568, x l2 > = (0-39212, 1,-0-82171) 

A <3) = 0-012093, x< 3 > = (0-36738,0-67764, 1) 

A 111 . x'»>being given by (S.43) and A 12 ’, x'-’-by (8.49). The best check on these results 
is provided by verifying that the relation* between characteristic vectors and 
between characteristic value's arc satisfied. 

Sine.- the characteristic vectors have been determined independently of one 
another, they can be cheeked by verifying that the orthogonality relations are 
satisfied, rin* results (S.f>0) j.rivc 

- i, — o-ooooi, 

J 

= — 0-00000 St 

which differ from zero by amounts within the tolerance for rounding errors. 


(8.50) 


V 


V 
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The sum of the characteristic values should be equal to the sum of the diagonal 
elements, which is -25 in this case, and this relation is satisfied exactly to three 
decimals. And since the elements of the matrix are integral, their determinant is 
integral, the product of the characteristic values should be integral. For the results 
(8.50), A ,l, A 12 , A (3) = —1-00007, which differs from —1 by an amount within the 

tolerance for rounding errors. . , 

A further check is provided by building up the matnx A from its characteristic 

vectors and characteristic values according to formula (8.41); thus, however does 

not give a good cheek on X™, since its contribution is multiplied by the sma 

nTm:\ i) The ratio |A<»/A«>| of the greatest and least in magnitude of thc cha™'- 
terUtic values is about 2300, ami the large sire of this quantity is an lt d cation 
of the degree to which the equations Ax = b. with this matrix A arc ill-con¬ 
ditioned. This character of A has not, however, introduced any difficulty in finding 

^“e^Sit'v^d vectors are found in succession, the lost ,in 

- "orthogonal to x™ and x» is formed by 

taking X = (x-/x, x< l) -/i 2 x< 2 >) ( 8 - 51 > 

and determining p„ p, appropriately. Since x<*>.is orthogonal to X™ it follows that 
the required values of fix »/x 2 aro 

p, = (2 ( ^' )! 

X should then be multiplied by such a factor that its groatest eomponent hi 1 and 
tl.o process repeated, as a check and to remove effects of rounding errors 

PO mho elements of A are integral, the following alternative process can sometimes 
be used to determine A™. As soon as A 1 " and A B > are found to the accuracy 

= — 27-969±0-00l, A ,2) = 2-957 ±0-001, 

it follows, from the relation 

A< 3 » = sum of diagonal elements = —25, 

that A< 3 > lies in the range A‘ 3 > = 0 0 12 ± 0-002 
and hence A<»A«'A'»> = -0-99±0-17. 

But since the elements of A are integral, this product must be integral, so must 
bo—1. Henco A' 3 ’= — 1 /A' ,, A ,:) = ±001209. 

(iii) The purification process can be use.! J° 7 )“*followH 1 ?.“^»fi7ui|>lieltion 

approximation to A' 1 ', x'”. V rorn ^‘ * J. x , 2 , fro , n a trial approximation 

by (A- 31) will be effective in remoung multiples 01 x lSH61 W o could 

to x«h Hence after the second step in the example as worked in * 8.61 

proceed as follows: ^ ^ _ _ 30 .930(1. -0-44.78, -0-06531). 

A (l, —0-44578, —0-06.31) = -27-9689(1,-0-44578.-0 00531), 
reaching the result in two fewer steps than in § 8.61. 
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This example illustrates some possibilities of Richardson’s purifica¬ 
tion procedure; for developments of the idea, and other examples, 
reference should be made to Richardson’s paper. 

8.7. Relaxation process for characteristic vectors 

Another method of determining characteristic vectors is a form of 
relaxation process. 

If l is any given number and we try to find by a relaxation process a 
solution of the equations 

(a n —J)a?i+ a 12 x 2 +a jz j'z f... = 0 \ (g 5JJ) 

U' 2 i 3-1 d - (^22 — 0*^2 d - ^23*^3 d~ ••• == 0 etc. I 

in which not all the x/s are zero, then unless / is a characteristic value, 

it will not be possible to reduce all residuals to zero. But if the x/e are 

components of a characteristic vector, corresponding to a characteristic 

value A, then . . . , .. .. 

(Uji —/)a-i+ai 2 T 2 -f« 13 T 3 4-... = (A— l)x lf 

®2l*l-M°22— 0-To-f «23- r 3+— = (A — l)X 2 , 


so that the ratio [(residual of the jth equation)/*,] will be the same for 
all values of j. 

If therefore wo fix one of the .r,’s at a non-zero value, and in the 
relaxation process aim not at reducing all the residuals to zero but at 
making the residual of each Jth equation proportional to x } , then a 
change of /. forming an improvement in the approximation to A, will 
have the effect of reducing all the residuals simultaneously. 

If an approximate characterist ic vector x can be estimated on inspec¬ 
tion of the equation, then an approximation A to the corresponding 
characteristic value is given by formula (8.37), and this can be taken as 
the best available value, at this stage of the calculation, to l. 

Exam plr: ( on-i< I r njjain i ho matrix usocl in the examples in the previous sections, 
nnmolv 

/-23 11 1\ 

A ( ‘1 zl -?)• 

Inspect .on suggests that x (2. -1,0) is on approximation to a characteristic 
vector. For tins x. f i»• • vartnr Ax is 

Ax (-57.25.2) 


a? id 


Wit!. I 


1' 

A / ■ 


131*'-, = — 27-S. 


2. S tli«» rn;«tri\ <>f tin* rofUioionts uf i-ipiations (8.53) is 

/ is n l \ 

a n In 2 is —2 

\ 1 — 2 2S S / 
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and the beginning of the relaxation process, keeping x, fixed, is as follows: 


Operations 


0 

0 


x- 


1 

0 


0 

l 


11 

1 


It.. 

24-8 

-2 


*3 

_o 

28-8 




x 2 

*3 

Hi 

r 3 

K, 


Relaxation 

2 

-1 

0 

— 1-4 

-2-8 

4-0 


table 



-014 

-1-54 

-2-52 

— 0-032 



0-1 


— 0-44 

— 0-04 

— 0-232 




0-01 

-0-43 

-0-06 

0-056 




0-01 


-0-32 

0-188 

0-036 



2 

-0-89 

-0-13 

-0-32 

0-188 

0-036 

Checked 


Although the residuals are not very closely proportional u, u»« > — - ( 
they are now of such signs and magnitudes that a change of l which would reduce 

It, to zero would also decrease |Jt,| and |i? 3 | considerably. 

When a stage such as this has been reached, it is best to calculate a new appro*.- 

mation to A; in this case, with 

x = (2, -0-89, -013), 

Ax = (-55-92, 24-93, 3-66), 

»34 50/4-8°90 = -27-908 

jk 1 i 

and with this value of/, the residuals, to three decimals, are 

0-016, 0-039, 0-014; 

as expected, these values are substantially smaller than those at the end of the 
relaxation table above. A further relaxation can then be earned out starting from 

A similaTproccss eon be carried out for the other characteristic vectors in order 
of decreasing |A|. For these, however, it is necessary either to obrnmatecaCone 
from the matrix as it is calculated, as ... §8.01. or to ensure that each as .t » 
calculated is made orthogonal to all those already determ,nod, as in § 8.O.. 

Characteristic values of linear ordinary differential equations with 
two-point boundary conditions can be obtained by a combination ot 
this technique with the replacement of derivatives by finite differences 
as in § 8.53.f 

t See L. Fox. Proc. Camb. Phil. Soc. 45 (1948), 50, § 8. 
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9.1. Solution of algebraic equations 

By an ‘algebraic’ equation is meant, in this chapter, an equation 
f(x) = 0 not involving derivatives or integrals of f(x), and of which a 
solution is a number, as distinct from a differential equation of which a 
solution is a function of the continuous variable x. It does not imply 
that the function /(.r) whose zeros are to be found is an algebraic function. 
For example the equations 

x 3 + 5a-=—3.r—2 = 0 

and e x sinx = 1 


are both ‘algebraic’ in this sense. 

/(.r) being a given function of x, the problem of linding the roots of 
/(•*•') = 0 is often best dealt with in two steps, the first concerned with 
locating the roots roughly, to two or three significant figures, and the 
second with improving these rough values. 

The solution of an algebraic equation can be regarded as a process of 
inverse interpolation, for if f(.v) is tabulated as a function of x, then the 
determination of the value of x for which/(.r) has any given value, of 


which zero is 
interpolation 


a special case, is just, the situation with which inverse 
is concerned. Once a solution has been located approxi¬ 


mately, tabula! ion of the function in the neighbourhood of that solution, 
followed by a process of inverse interpolation, is one way of determining 
it more exactly. Vnother method is to use an iterative process; this is 
considerslint; 9.3. 


9.2. Graphical methods 

l se oi graphs is often a valuable method of locating approximately 
the roots of an equation f(x) u. Either the function/(.r) itself may 
la- graphed and its intersections with the x axis determined, or the 
equation may he written in the form /,(.»•) --- f,(x). and its roots deter¬ 
mined by the intersection of the graphs of // - = J\{.v) and // =. /.(.»•); it 

may be possible to avoid some calculation by this process. In some cases 
graphs using some argument other than x. such as log.r or 1 x. may be 
useful. The best procedure will depend on the form of the function 
J(x). and it is dihicult t<> lay down any general rules. 
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The following examples give suggestions for handling some kinds of 
equations: 

(i) Equations of the form /(x) = x n + ax + b = 0. Use graphs of y - x an 

y = — (ox-f b). 

(ii) Equations of the form 

f(x) = x n + a i x n_l + a 2 x n ~* + .••+° n -i x-\-a n = 0. 

Evaluate f(x) directly (see § 3.2) or by building up from its differences (see § 4.42) 
for, say, * = -5(0-2) + 5; for |x| > 4, say, Ukc y = \/x and evaluate 

F(y) = o n y n + o n _ l y n ~ 1 + — + o 1 y-\-a 0 . 

Use graphs of J(x) against x and F(y) against y. 



(iii) cl* sin x = 1. Write this as sinx = c »* 
y = c'l* with y = sinx (see Fig. 13). This 

el* sinx. 


, and determine the 
avoids calculation 


intersections of 
of the products 


9.3. Iterative processes . _ n 

By an iterative process is meant one in which the equation f(x) 

is expressed in the form x __ F(x), 

and we try to find a solution by constructing a sequence {*„} by the 
relation x lltl = F(z n ). <<J1) 

If;^he degree £££. ^ 

degree of accuracy. 

Let x ^ X be the solution of the equation, and let 

v it (9.^) 

x n = A -K„ 
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so that $ n is the error in x n , regarded as a solution of the equation. An 
important feature of an iterative method is the way in which this error 
varies with the number n of repetitions of the iterative process. This can 
be examined by expanding the right-hand side of (9.1) in a Taylor series. 
Then, since x = X satisfies x — F(x), it follows that 

£n+i = a i£n+a 2 £*+a 3 $*+..., (9.3) 

where a k — F ik) (X)/k\. 

If a x ^ 0, then the errors £„ of results of successive repetitions of the 
iterative process are ultimately related by 

£n+l == ®l£n» £n+m £n> 

in order that the process should converge, |aj = |jP'(X)| must be 
less than 1, and the magnitude of the error then decreases exponentially 
with n increasing. This means that the number of additional correct 
significant figures obtained from each repetition of such a process (or, 
more often, the number of repetitions required to obtain each new 
correct significant figure) is the same, however many figures have been 
obtained. Such a process is called ‘first-order’. 

But if a x = 0, a 2 ^ 0 in (9.2), then the successive errors £ n are 
ultimately related by 

= «a£n. OF a 2 £n+m = («2 £»)*"*. 

where a 2 = hF"{X). The number of correct significant figures is approxi¬ 
mately doubled for each repetition of the iterative process, so that the 
better the approximation of x n to X, the easier it is to improve it further. 
Such a process is called ‘second-order’, and once a fair approximation 
to r —- A' has been attained, a second-order process is very greatly to 
he preferred to a first-order one; but it must be started from an approxi¬ 
mation good enough to ensure that |a 2 £ 0 | <1. It will be shown in 
§9.32 that from any first-order process it is possible to derive a second- 
order process. 

If « l -- (i, a., = 0, </ 3 ~ 0 in (9.2), then the successive errors £ n are 
ultimately related by 

* V 


£»♦> ■= «3^u. or = (<4 £„)*"; 

such a process is called ‘third-order’. The formula for a third-order 
process is usually more complicated than that for a second-order process 
tor the same equation, and the convergence of a second-order process 
is already so fast once a good approximation has been obtained that the 
advantage of still quicker convergence obtainable from a third-order 
process may be more than offset by the more complicated formulae 
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which have to be evaluated for each repetition of the iterative process, 
and third-order processes are not much used in practice. Second-order 

processes, however, are widely used. 

9.31. Examples of iterative processes 

(a) Newton's process for a square root 

An important example of a second-order process is one for a square 
root, usually known as ‘Newton’s process’. If b is the number whose 
square root is required, this process consists of forming the sequence 

{*,.} defined by * n+1 = J[*„+(6/*„)]. 

For this process, X - - ft*, and F(x) == )[*"t (ft/*)]. 80 that 

F'(x) = i(l — ft/* 2 ), F'(X) = 0, 

F'\x) = ft/* 3 , F"(X) = 1/X, 

so that a, = 0, a 2 * 0 in (9.3), and the process is second order- 

As an example of the application of this process, consider the eralua- 

tion of V12, starting from x 0 = 2. 

x 0 =2, — i(2 + ^ 2 -) = 

x 2 = i(4+*J) = 1 = 3 ' 5, 

*. = !. x, = Kl+¥) = Si = 3-4643, 

* 3 = 3-4043, 12/*, = 3-46390,32, *, = 3-46410,16, 

z 4 = 3-46410,10, 12/*. = 3-46410,16303, * 5 = 3-40410,16151. 

them in decimal form. ■ . • , heen taken as a 

(ii) In this example, an unnecessarily bad a very poor 

starting-point to illustrate the convergence of the process f . 3 fairly 

value of x„. X more important api9ieation^sthe^impro^ement^ofan^alread^Ja^ 

fbu^fig^res esm alwaytrtraobtainedwithout g^ve’fiftiren^figure^i ^t 

of Newton’s process will give eight figures, and another w.ll give b 

least. 

Newton’s process is not the only sccond-order one for a square root, 
another is given by . 

;Cn+1 = *„(36-*S)/26 = *„[l + (fc-^)/ 26 ]’ (94) 

This process does not converge as fast as Newton’s, since for a given 
value of ft. the value of u, in the series (9.3) is.greater. But. 1^ one 

feature which may be an advantage, namely t at e ■^ I ^ tion , )roc ess 
constant instead of being different at each stage o 

as it is in Newton’s process. 


*i=4 


6SS3 


O 
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(6) The Newton-Raphson process 

A general second-order process for the solution of f(x) = 0, at a point 
not in the neighbourhood of a maximum or minimum of f(x), is one 

given by *» +l = x.-fMIf'M- (9-5) 




This is known as the Newton-Raplison })rocess; Newton’s process for 
a square root is the special case of it for the function f(x) = x 2 —6, and 
(0.4) is the special case of it for f(x) = 1 —b'x 2 . For the general Newton- 
Raphson process, 

F(- r) = x-f(x)',f\x), F'(x) = -/(*)/>)/[/'(-r)] 2 , 

and since /(A') = 0, it follows that F'(X) = 0, so the process is second 
order. 

Expressed in terms of the graph of f(x), the Newton-Raphson process 
is equivalent to linear interpolation along the tangent to the curve 
y = /(.r) at .r„ (see Fig. 14). 


(f ) The ‘ rule of false position ’ 

Anotin r iterative process is equivalent to linear interpolation along the chord 
joining [r 0 ./(x 0 )l to [j-„./(r„)] (see Fig. 15). This gives 

= ^-(^„-io)/o/(/,-/o) = (*o/n-*n/o)/(/n-/o)- 
'J hLs method, however, is only first order, though if r 0 is a fair approximation to X, 
the coefficient n y in (9.3) is small; successive errors are related by 

£n + i = h^o-N)f'(X)$Jf'(X), 

approximately. It hus the advantage that it does not require the evaluation of/'(x). 

A disadvantage of methods such as the Newton-Raphson and the 
method of false position is that they involve the evaluation off{x) and 
/ (.') at a number ot values of x which, though systematic in the sense 
that each is calculated irom the previous one by the same formula such 
as (‘>.f>), are irregularly spaced, and such a set of numbers is difficult to 
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check adequately. An advantage is that a mistake in an intermediate 
value of x n does not affect the final result; it is just equivalent to starting 
a new iteration with this erroneous value of x n as ar 0 - But this does not 
eliminate the possibility of a mistake in the last repetition of the iterative 
process. Tabulation of f{x) at equal intervals of a: followed by a process 
of inverse interpolation is a process which provides more, and simpler, 
checks against occasional mistakes. 

Example : To find the root of * tan a: = £ which lies between x = 0-6 and 0-7. 

(a) By tfie Newton-Raphson process 

There are several forms in which this equation can be written, for example: 

/ (x) = x tan x— l = 0; f(x) = 2x — cotx = 0, 
f(x) = 2 xsinx —cosx = 0. 

The third of these will be adopted, as it gives the most convenient formula for/ '<*), 


namely 


f'(x) = 2xcosx + 3sinx. 


Starting with x 0 = 0-6, sinx 0 = 0-5646, cosx 0 = 0-8253, we have 


/(x 0 ) = 2x 0 sin x 0 - cos x 0 = -0-1478 
/'(x 0 ) = 2x 0 cosx 0 + 3 sinx 0 = +2-68421* 


/(• r o)//V o) = — 00551, 

Xl = 0-6 + 0-0551 
= 0 6551, 


x x = 0-655, sinx, = 0-609159, cosx, = 0-793048 
/(x,) = 2x,sinx, — cosx, = +0-004950 | 
/'(x,) = 2x, cos x, + 3 sin x, = 2-86637 /’ 


fMI/'M = +0001727, 

x 2 = 0-655 —0 001727 
= 0-653273, 

x. = 0-653273, sinx 2 = 0-607788. cosx, = 0-794099 

/(x.) = 2x, sin x,-cosx, = +0 000007, , /(*,)//'(*.) = +0 000002,, 

/'(x,) = 2x, cos X, + 3 sin x, = 2-80097 )’ x, = 0-0o3270. 

»*«.:■(i) The first approximation x 0 = 0 0 is a rough one and 

of sinx, cosx are adequate at this stage; more figures are used later when 

accuracy of x„ has been improved. . , f , vn i u „ 

(ii) For the second stage of the iteration, x, is taken as O-Ooo mstom 

0-6551 obtained from thefirststago. It is not to be expected t j^les tableVwith 

of this value will bo correct, and the rounded value x x - O-Ooo cn . 

interval 8x = O-OOlf to bo used without interpolation, tor t 10 i b 

ever, interpolation in the tables is necessary. , , . Iise 

(iii) For the third stage (and later stages, if any) it would o a I • 

/'(x,) instead of recalculating /'(*„) for each new voluo such 

method formally only first order, but the coefficient a x m (9.3) >_ “ 

a case that the convergence of the first-order process is adequate for piact.cal 

(6) By inverse interpolation . 

To solve the equation by inverse interpolation, it is most com emc n 

i in the form — 2x—cotx = 0 

t Chamber*'* 6-Figure Table,, vol. 2 (1649), for example. 
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as this avoids the calculation of any products and involves the least reference to 
tables. Evaluation of/(x) to two or three decimals at intervals 005 or 0-02 in the 
range x — 0-6 to 0-7, as might be used for a rough plot, locates the root as lying 
between x = 0-65 and 0-66, and taking 0 01 intervals we have the following table: 



cotx 

f(x) = 2x — 

cotx 

S 2 / 

0-64 

1-343104 

-0-063104 

47668 


0-65 

1-315436 

-0-015436 

46950 

— 718 

0-66 

1-288486 

+ 0-031514 

46265 

-685 

0-67 

1-262221 

-f 0-077779 




Inverse interpolation for /(x) = 0 at z = 0-65 + 0(0-01) gives 

e = 

9 = 0-32878— -B ,U) (0). (0-02988), 

and iterative solution of this equation gives 0 = 0-32715, x = 0-653271 5 . 

Alternatively, 0-002 or 0-001 intervals could be taken between x = 0-65 and 
0-66; for example 



cotx. 

/(x) = 2x — 

cotx 


0-650 

1-315436 

— 0015436 

9447 


0-652 

1-309989 

-0-005989 

9418 

-29 

0-654 

1-304571 

+ 0-003429 

9389 

-29 

0-656 

1-299182 

+ 0-012818 




and linear interpolation is now adequate to give five decimals in x. 

Note: The two methods may be combined; for example after obtaining the approxi¬ 
mation x x by the Newton-Raphson method, the approximation to the root may bo 
improved by tabulation at 0-001 intervals in the neighbourhood of x — 0-655, 
followed by inverse interpolation. This avoids the interpolation in tables which 
has to be done if the Newton-Raphson process is continued. 


9.32. Derivation of a second-order process from a first-order 
process 

If it is known that an iterative process is first order, this knowledge 
enables a better approximation to the solution to be obtained by an 
application of the process of ‘exponential extrapolation’ (see § 3.4(a)). 
if t lie first term in the expansion (9.3) were the only term, then we would 

h ‘ lVe &/fi = £i/£o = "i» that is, £ 0 £ 2 = f; 

exactly, so that X would be given by 

(x.,-A')(.r 0 -A') = (a^-A) 2 . 

Unless the higher terms in (9.3) are negligible, this will not give exactly 
the value of A. but an approximation, say A'*, to it: 


A 


*2 


__ r (*,-.!•» ) 8 

• r 2 ~ i r -< o 2 •*■•> — 2.r x + .r 0 


(9.6) 
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(see eq. 3.12), which will usually be a substantially better approximation 
than z 2 . We can then repeat the process starting with x 0 = X*. 

In general, let X* +1 be the result of this process, starting with :r 0 - X n . 
Then it can be shown that the process of forming the successive values 

of X* is second order.f 

Example: To solve the equation **_6x + 2 = 0 by writing it in the form 

* = 6 — (2/x) 

and using an iterative process. # /0/ x A nrl 

In this case the function F(x) of § 9.3 is F(x) — 6 ( lx), ( ) / 

though the solution x = X is not yet known, wo can be sure that F (*) ** not zero 

there, hence the iterative process x n+ , = 6-(2/x„) is first order. 

Starting with X* = 3 we have 


x — 3 , x, = x 2 = 


8 


Then, from formula (9.6), 

x 2 —x l = YJ-, 
45 


x 2 — 2 x ] -f- x 0 = 
,2 17 

3 


_ __ *a 

2 4 * 


V* _ (7/24)-_ = w = 5 . 66667> 

-^-1 O 


8 — 49/24 

Then with x 0 = X* = V. we have p f ro . 

= 6-2.3/17 = 5-64706, a*, = 6-2/5-64/06 - .>-64584, 

then from formula (9.6) again, 

z t —x l = -0-00122; x a -2x, + x 0 = 0-01839; 

v * - fi4 r, s4 (O-OOlgg)* = 5-64584-0-00008 = 5-64576 
A* = 5-64.184- 0 . 01H39 

4 

which is already correct to six significant figures. 

9.4. Multiple roots and neighbouring roots 

Particular care is necessary when the coefficients in the equate. are 
in the neighbourhood of values for which the equat.on has mult,pie 
roots. The values of the roots are then part.cularly scnsit.v e the 
values of the coefficients and to rounding errors, io take a snnple 

example, the equation 

a - 2 _g z _j_ ifi-00 = 0 has a double root x — 4, 

a . 2 _g ir _|_ 10-01 = 0 has no real root, 

a: 2 _g a: _j_ 15-99 = 0 has roots 3-9, 41, 

so that a change of less than one part in 1000 in the constant term affects 
the roots by one part in 40. The situation is clear here but may not be 
when the equation has other roots or involves transcendental 
If a repeated root is suspected, either from the results of lus process,^ 
from the graph, then careful numerical evaluation of the y 
be carried out in the neighbourhood of the suspecte, rep 

t Soo D. R. Hortroo, Proc. Camb. Phil. Soc. 45 (1948). -30. 
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Two (or more) close but not equal roots may be more troublesome than 
a true repeated root. If two close roots are suspected from examination 
of a graph or on other evidence, there will certainly be a root of f'(x) in 
this neighbourhood, say x — x m , and it is best to determine this first, 
and to evaluate f{x m ). If this has the same sign as f(x) at neighbouring 
values of x, then there is no real root; if f{x m ) has the opposite sign to 
f(x) at neighbouring values of x, then there are tw'o real roots. Since 
f'(x m ) = 0, Taylor’s series for f(x) in the neighbourhood of x m begins 

/(*) + 

and if (x—.r, n ) is sufficiently small (and f\x m ) is not too small) 

= ±[2 {/(*)-/(*J}/r<*J]‘- (9-7) 

This, if not already accurate enough, will provide starting values for a 
further approximation to these roots. 

The calculated values of these roots will be very sensitive to rounding 
errors; if f(x) is given by a formula which can be evaluated to any degree 
of numerical accuracy (for example, by a polynomial), the roots can 
be evaluated to any accuracy required; but if the evaluation of f(x) 
involves reference to tables, the accuracy of the calculated values of the 
roots mav be small. 

Example: To find the smallest positive root, of xcos 2 x — 0-4115. 

A graph shows that/(.r) = xcos 2 x has a maximum of aholit 0-41 in the neighbour¬ 
hood of x = 0 65. 

For this function, /'(x) = — cosx(2.rsinx — cosx), and this has a zero at 
x m = 0-65327 (see § 9.31, oxample), where/(x) = 0-411949 ; also 

/"(Xm) = -cosx m (2x in cosx m + 3sinx m ) = -2-272. 

So /(x) — /(x m ) = 0-4115-0-411949 = -0-000449, 

and the smallest root of xcos 2 x = 0-41 15 is approximately 

0-65327 — [2( — 0-000449)/( — 2-272)]* = 0-65327-0-01988 

= 0-63339. 

Note : The fifth decimal of this value of x is not determined to ± 1 by the sixth 
decimal of/(x) or/(x m ). 


9.5. Special processes for special types of equations 

The methods so far considered have been general methods applicable 
to any kind of algebraic equation, in the sense explained in §9.1. For 
some special kinds of equations f(x) = 0, and particularly for those in 
which/(x) is a polynomial in x, there are special methods. 

Polynomial equations, like linear simultaneous equations, are met in 
contexts of two kinds. In one the coefficients are all known exactly and 
are usually integral; in the other they are only known to within a certain 
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tolerance because they are results either of observations subject to 
experimental errors or of other calculations which are subject to rounding 

errors. - 

In the latter case it may be important to know the range of uncertainty 
of the solution arising from the tolerance in one or more of the coefficients. 

If the polynomial is 

f(x) = a Q x n +a 1 x n -' l +a z x n - 2 -\-...+a n - x x+a n = 0 (9.8) 

consider the first-order variation of a root * = X with one of the co¬ 
efficients, a k . If the root X changes by AX when a H changes by A a k> then 

/'(X)AX+X"-*Aa* = 0, 

that is, AX = -[X*-*lf'[X)]*a kt 

and for changes (not necessarily equal) in all the coefficients 

AX = -[2 (X n "*Aa*)]//'(X). (9.9) 

This shows that the roots are particularly sensitive to the values of the 
coefficients in the neighbourhood of a stationary value of the function. 


9.51. Quadratic equations 

It has already been pointed out that direct evaluation of the standard 
text-book formula for the roots of a quadratic equat ion + bx-\-c , 

namely x _ [ — b±{b 2 — 4 ac)*]/ 2 a, 

is not always the best way of determining numerical values of the roots 
of a quadratic and particularly not if the ratio of the roots is large 


(b 2 > 4ac). 

A better practical method in many cases is an iterative process 
on use of the relations 


based 


x l r^-x 2 =—b/a, x l x 2 — c/a, 

where *. and are the two roots. If *, is the root of greater modulus, 
then successive approximations to the roots can be evaluated by using 

the formulae ^ = _(6/a)-.r 2 , * 2 = (c/a)/*! (9,l0) 

alternately, starting from the approximation * 2 = 0 if no other is easily 

available. , „ . 

This process, though only first order, can be carried out so easily, a. 

when 6 2 > 4ac it converges so quickly, that it is unnecessary o re in 

further. If b 2 is not considerably larger than 4ac, it may be convemen 

to use a second-order process derived from this first-order process as 
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explained in § 9.32. Elimination of x 2 between the two equations (9.10) 
glves x 1 = — {bfa)—(c/dlfa 

which is the general expression of which the example given in § 9.32 is 
a special case. 


9.52. Cubic and quartic equations 

A cubic equation with real coefficients has at least one real root, say z t i if it is 
determined, division of the cubic by (x—x t ) gives a quadratic which can then bo 
solved by the standard formula or by iteration. For determination of the real root 
or roots the general methods of the previous sections will often be best. 

There are also special methods available which depend on reducing the cubic to 
a standard form. If the cubic i3 


ax 3 -j-&x 2 -f-cx+d = 0 


then the substitution y — ft(x-\-b/3a) reduces it to a cubic in y without a y* term; 
then either the coefficient of the term in y is zero, or a real value of ft can be chosen 
so that the ratio of the coefficients of the terms in y and in y 3 is either -f 1 or — 1. 
Thus any cubic can be reduced to one of the forms 

y 3 +D = 0, y 3 -\-y + D — 0, y 3 —y + D = 0; 


the solution to an equation of the first of these forms can be found directly from a 
table of cube roots, and tables of the roots of the equat ions of the other two forms 
have been evaluated (for details, see the Index of Mathematical Tables). 

If in til* 1 substitution y — ft(x b!3a), ft is chosen so as to reduce the equation 


to the form 


4y 3 ±3 y-D = 0 


the further substitut ion y — sinh u (if the sign of the middle term is +), y = cosh u 
(if the sign of the middle term is —, and D > 1) or y =■ cos u (if the sign of the 
middle term is —, and D < 1) can be used to reduce it further to 


sinh 3u = D, cosh 3a = D, or cos3u = D 


respectively and the solution found from tables of hyperbolic or Circular functions. 

The solution of a quartic equation can be reduced to the solution of a cubic 
equation, but in most cases it is better to use more direct methods. 


9.53. Polynomial equations 

For solving polynomial equations there are available a number of 
special methods.| Various theorems in the theory of equations can be 
used to determine how many roots lie in various ranges of x, and various 
special methods are available such as Horner's method or a method 
known as ‘root-squaring’ which depends on forming an equation whose 
roots are some high power ot the roots of the equation to be solved. 

I he root-squaring method, however, only gives the magnitudes of 
the roots and not their signs, and some evaluation of f(x) is necessary in 
order to determine their signs. Also both methods, as usually presented, 

+ I* or a survey, with particular reference to equations of high degree, see F. W. J. 
Giver, I'/til. Trans. Hoy. Soc. A. 244 (1952), 385. 
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are deficient or lacking in current checks, and their results should always 
be verified either by substitution in /(x) or by evaluating f{x) at a 
numerically convenient set of values in the neighbourhood of each root 
followed by interpolation. The root-squaring method in particular 
offers too many opportunities for mistakes for any alleged root to be 

accepted without such investigation. 

Some evaluation of f{z) is therefore required in any case, and it seems 

better for most practical purposes to use the general methods already 

considered, namely the use of graphs for approximate location of the 

roots followed either by evaluation in the neighbourhood of each root 

and inverse interpolation, or by an iterative process,f rather than to 

use the special methods available for polynomial equations. 


9.54. Repeated roots 

Repeated roots of polynomial equations can be located by finding t le 
highest common factor of /(*) and /'(*>. If the coefficients m the 
equations are known exactly (when they will usually be Integra ), 
repeated roots can be identified with certainty if the H.C.F. process is 
carried out exactly without any rounding off. Otherwise there can 
generally be no certainty whether the equation has a repeated root or 
two (or more) very nearly equal roots. However the HX.F. process 

may well establish the absence of a repeated root. 

With polynomial equations repeated roots should be removet >\ 
dividing/(x) by the appropriate product of repeated factors before the 
determination of the remaining roots is started. 


9.55. Division of a polynomial by a quadratic 

A convenient way of finding the complex roots of a real poly no mm 
equation with no real roots is to express the polynomml as a product of 
real quadratic factors. In one process for doing this it ,s necessary to 
carry out a number of divisions by successive approximations to a i u 
quadratic factor, and it will be convenient first to consider a numerical 

process for carrying out this division. 

Let a n X n +a.x n - l -\-a 2 x n - 2 +... + a n -i*+ a n 


be the dividend polynomial, and 

x-\-d 2 

the divisor quadratic. Then we want to find a quotient 

q 0 x r '-*-\-g l x n - 3 +^*"- 4 +- + 7 w -»* + 7--. 

t For an iterative process for complex roots of polynomial equations, 
son, Journ. Mailt, and Phy». 28 (1949), 259. 


polynomial 


hoo 1*. A. Samuel- 
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and a remainder such that 

a 0 x n +a x x n ~ l +...-\-a n 

= {x 2 +d 1 x+d 2 ){q Q x n -*-\-q l x n - :i +...+q n _ 2 )+r 1 x-\-r 2 . 

Multiplying out, comparing coefficients, and solving for< 7 0 , q v ...,q n - 2 > r v r i 
in succession we have 


a o — (7o» 

«i = 

a 2 = (72 + ^1 (7l + d 2(7<)’ 
a 3 ~ 5'3 + ^l?2 + d 2 5 r i» 


(7o = «o> 

(7i ~ °i ^i(7o> 

(72 == a 2 di (7i d 2 (7o' 

(?3 == °3 *7 2 ^2?1» 


a n -2 — (7n-2 + d l(7n-3 + < *29 r /«-4> 

«n-l = »’l + rfig , »-8 + rf2S r «-3. 

a n = ; ’2“i _c ^2 9 , /i-2» 


(7>i-2 = a n-2 ^1 (7/i -3 ^2(7 >i-4» 

r l = a n-i— ( hQn-2~doq n -3, 
r 2 — a n d 2 (7n-2- 


The numerical work can conveniently be arranged in the following 
scheme: 


Coefficient of 


x" _l 

ar" -3 . 

X 4 

X 

1 

1 

• 

"0 


a 2 

• • a n -2 

"n-l 

a n 

i 


7o 

— cliQl . 
7 0 _. 

• • <7/1-3 

• * “^2 

r~ d -2 <7n- 3 

^2 <?n -2 

Sum 



' ?2 





On the left the numbers 1, — c/j, — d 2 are written in three successive 
lines. In the first row are written the coefficients of the dividend poly¬ 
nomial. As each coefficient 7 ; in the quotient polynomial is determined, 
it is multiplied by —c 7, and — d,, the product —d x q j being written in 
f ' ,tf second ,inc in the column next to that containing 7 < Jt and —d z q j in 
ihe third line of the column next but one, as shown by the arrows." The 
sum of the entries in the column containing a f is the corresponding 7 
This process, in which the division is started at the highest power of 
c and the remainder is of the form r,.r-fr 2 . will be called ‘forward 
division’. A similar process can be carried out starting at the term 
independent of x and proceeding from lower to higher powers of .r, with 
a remainder K, *»--*+*,,*« This will be called ‘backward division’. 
In this process it is convenient to take the divisor as 1+ D l x+D 2 x 2 , 
and to place I), on the right and work from right to left, thus: 


Coefficient of x n 

.c"- 1 | 

.r"- ! . 

X 2 

X 

1 


«0 



• • a n—2 

a n-l 

a n 

1 


“AG. 


• &lQn-\ 



-Dy 

~d 2 q. 






-D t 

Ko 

1 

• 

• • Vn-2 



Sum 
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Example : 


To divide x 4 + 5* 3 + 1 2x 2 + 14x + 8 by x 2 + 2x 


4 and by 1 + "b l £ “ m 


Coefficient 

of x- 4 x 3 x 3 

x 1 

X 4 x 3 

X- X 1 

1 1 5 12 

_2 -2 -6 

14 8 

-4 

^12__-8 

1 s 

___ \ \ 

12 14 K 1 

^-3^-12^ -l 

Sum 1 3 2 

Quotient 
x 3 f 3x4-2 

"-2 0 
Remainder 
— 2x 

-§ -i 

Remainder 
| -W-lx' 

—^5 * 2 8 Sum 

Quotient 

8 -f 2x + 5x 2 


9.56. Real quadratic factors of a polynomial 

If some of the roots of a real polynomial equation f(x) = 0, are 
complex, it is best first to determine the real roots x x , x 2 ,..., x k by the 
methods* already considered, and remove them by dividing f{x) by 
(x-x x ){x-xi)...(x-x k )\ repeated roots, if any, should be identified as 
explained in §9.54, and taken out with the corresponding multiplicity. 
This process provides a good check on these roots, since if the re¬ 
mainder in the division differs from zero by a greater amount than the 
tolerance for rounding errors, one of the roots must be in error. 

The quotient will be a polynomial F(x) with no real zeros, and its 
complex zeros can be determined by finding its real quadratic factors. 
We take a trial quadratic D 0 - x*+b 0 x+c 0 and find the quotient C> 0 
on forward division of F(x) by D 0 (the remainder is irrelevant). Then we 
find the quotient on backward division of F(x) by Q 0 (or a multip e 

of it), write 

A = Q*l( coefif. of x 2 in (?J) = * 2 -f byX+Cy, 

and repeat the process with A in place of A- This provides an iterative 
processf in which the successive quadratics 

D 0 = x 2 +b 0 x+c 0 , 

% 

A = X^ + byX + Cy, 


A = .t 2 + b 2 x+c 2 

converge to the quadratic factor corresponding to the roots of smallest 
modulus of the equation F(x) = 0. The condition for convergence is 
that these roots should be of smaller modulus than any other roots ot 
the equation; one of the reasons for removing the real roots is to ensure 
that there shall be no real root of smaller modulus than any of the complex 

t n. Friedman, Co.. on Par. and A PP l. Mo„,. 2 (1»49). 19». -'or another 

iterative nrocctw bco A. C. Aitken, 1’roc. Uo.j. Soc. Ed,n. 63 (lJel). 174, 
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roots as the presence of such a root would make the results of this 
iterative process oscillate instead of converging. 

Example : To find the real quadratic factor of 

x* + Sx 3 * 12x 2 -j-14x-f 8. 

The first steps, starting from x 2 4-2x4-4 as trial quadratic factor, have already 
been carried out in the example of the last section. The quot ient of the first forward 
division is x- + 3x-f-2 = 2(1 -f $x-f £x 2 ); the factor 2 is taken out in order that the 
constant term in this divisor should be unity, and 1 -f §x-f Jx 2 used as the divisor 
in the backward division. For the present purpose the remainders are irrelevant, 
and the two pairs of columns in the centre can be omitted, so that thus far the 
working could be written 



Coefficient of x 4 x 3 x 2 x 2 x 1 

T j 1 5 \2 12 14 8 i 

-1-0 : —1-6 -5-44 -5-72 -5-20 -0-65 

_-F3 I -13 -152 -0-19 

; i 3-4 5-20 Ptb jTso 8 

-- 5-26(0-19 0-65 1) j= 4-76(1 1-85 1-68) 


(’oofTicimit of 

.r 4 x 3 

1 

X 2 

X 

1 

i 

1 5 

12 

12 

14 

8 1 

— I -85 

— 1-85 

- 5-83 

- 5-88 

- 5-62 

-0-702 

1-68 


- 1-68 

- 1-78 


-0 223 


1 3 15 

4^49 

4-34 

8 3jT 

8 


= 4 49(0-223 0-702 1) = 4-34(1 1-931 1-843) 

1 last three approximations to a quadratic factor are 

x--r 1-6x4.13, 

x 2 -) 1-85x4-1-68, 
x 2 -h 1-931x4- 1-843. 









NON-LINEAR ALGEBRAIC EQUATIONS 


205 


The process is first order, so we can use the method of‘exponential extrapolation’ 
(see §§3 4 (a) and 9.32) to estimate a better approximation from these. If we write 
the quadratic x- + bx+c, with suffixes 0, 1, 2 for these approximations, and use 

formula (9.6), we have 

& 3 _& l = 0 081, b 2 -2b l +b 0 = -0169. 

extrapolated 6 = 1-931 + ^ Q.jgg - = 1-970; 
c 2 _ Cl = 0163, c t -2 Cl +c 0 = -0-217, 

extrapolated c = 1-843 q. 2 i 7 ~ = 1-905: 

and the calculation can bo continued from these values in a similar way An 
alternative method of improving the approximation to a real quadratic factor Will 

be considered in the following section. 0 ;„nifipont 

Notes : (i) As illustrated in the working of this example, only ‘ 

figures need be kept at first when the approximation to a quadratic factor .s still 

only rough, and more kept os the calculation proceeds. r rtTOJor ,i 

fi) When applied to a 2„tl, degree polynomial, the quotient of the forward 

.... . , • i t i w . /o.,_ 21th decree and in the backward division we 

division is a polynomial of the ( 2 »- 2 )tn uegreo an A 

have to divide by this quotient. But the quotient of the backward division is a 
quadratic and is determined by the leading three terms in the d.v sor in 
backward division, so that the above process for division by a quadratic can still 

’’Vi^There is no accumulation of rounding errors, since at each stage the 
original polynomial is divided by the current trial quadratic factor. 

9.57. Second-order process for improving the approximation 
„ to a quadratic factor 

The following is an extension of the Newton-Raphson process to the 
improvement of an approximation to a real quadratic factor ol a leal 

polynomial f{x). 

Let (x 2 + 6x-f c) be an approximate quadratic factor am e 

, f(x) = (x 2 +bx+c)q(x) + rx+s, (9- 11 ) 

where q(x) is the quotient polynomial on division of/(a) by ('*+4*+e). 
and («+a) is the remainder. These can be found by the method 
§9.55. Differentiation of (9.11) with respect to b, for constant x am r 
gives the variation of the coefficients r and * in the remainder w.tl. 
variation of the coefficient b in the trial quadratic factoi. 

0 = ^ + bx+c)^Y Xq(X) + ^ X + ^b 

so that -( drldb)x-(Bsldb) is the remainder when xq(x) is divided by 
(l Uk +c) . Similarly differentiation with respect to c gnos 

r ds 


0 = (x 2 + bx-\-c) 


PTH-’+g) 




CC 
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so that — (drfdc)x — (ds/dc) is the remainder when q(x) is divided by 
(z 2 -f bx-\-c). These remainders can be found by the method of §9.55 
(the quotients are also found, but are irrelevant) so the partial derivatives 
of r and s with respect to b and c can be determined. 

If now changes A6 and Ac are made in 6 and c, the first-order changes 
in r and s are 

(S)“+(£h 6)“+6b 

and we want to chose A6, Ac so as to reduce r and 5 to zero, that is, to make 



These determine A6, Ac and hence a better approximation 

**+(6 + A6)ar+(c -f Ac) 

to the quadratic factor sought. The process can be repeated, and is 
second-order. 

Example: To improve the approximation x 2 -f- 1 -970x + 1 -965 to a quadratic factor 
of x 4 -j-5x 3 -+- 12x 2 -f 14x-}-8 (see example in previous section). 


Coefficient of 

X* 

X 3 

X 2 

X 


1 


1 

1 

5 

12 

14 


8 


- 1-970 

- 

-1-970 

-5-9691 

-8-0098 



- 1-965 



- 1-965 

-5-9540 

-7-9895 


!> 

1 

1 

3-030 

4-0659 

0-0362 

00105 


Quotient q( 

x) = X 2 

: 3-030x + 4-0659 

1 - 


s 


Coefficient 








of a' 3 

X 2 

X 

1 

X 2 


X 

1 

1 |j-?(x) = 1 

3 030 

4-066 

0 

9 (x) = 1 


3-030 

4-066 

- 1-970 - 

1-970 

-2-088 



— 

1-970 


- 1-965 | 


- 1-965 

-2-083 



— 

1-965 

1 1 

1-060 

0-013 

-2-083 | 

-- — 

1 


1-060 

2-101 


-(I) -(I) 


-(I) -(I) 


Hence A';, Ac are given by 

0-013A6 + l-OGOAc = 0-0362, 

— 2-083A6 -f 2-101 Ac = 0 0105, 

and solution of these equations gives A b = 0-0291. Ac = 0-0337, whence 

b = 1-970 + 0-0291 = 1-9991, 
c = 1-965 f 0-0337 = 1-9987, 

so that .v--- 1 999lx-f 1-9987 is a better approximation to a quadratic factor. 
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Actually the quadratic factors in this case are x 2 -\-2x-\-2 and x--f 3x+4 exactly. 
One application of this method has improved the approximation to the factor 
x 2 -\-2x-\-2 by a factor of about 30. 

9.6. Simultaneous non-linear equations 

For simultaneous equations in two variables the same general pro¬ 
cedure as for equations in one variable can be used, namely a graphical 
process for locating the roots approximately, followed by a numerical 
process for improving the approximation. 

Let the equations be 

A(x,y) = o, /«(*,y) = °- (9 - 12) 

If both of these can be solved formally for y as a function of x, or x as 
a function of y, then it is easy to draw graphs of y against x for each 
equation, and the intersections of the two graphs give an approximation 
to the solutions. If one or both of the equations can be solved formally 
for y as a function of x or vice versa, then one of the variables can be 
eliminated and the equations reduced to an equation in one variable; 
for example if the second of equations (9.12) can be sohed in the foim 
y == (f> 2 (x), substitution of this in the first equation gi\es 

F,(x) = A(x,<f> 2 (x)) = 0. 

There is no need to carry out the elimination explicitly in such a way 
as to exhibit F^x) formally as a function of x; all that is wanted is that 
y = <f> 2 {x) should be evaluated for a set of values of x, and that these 
should be substituted into the formula for /,(*,?/) for the corresponding 
values of x. This process carries out the elimination numerically without 

its having to be expressed formally. 

Example : sin x + 2 sin y = 1, 

2 sin 3z + 3 sin 3y = 0-3. 

It is most convenient here to solve the first equation for sin y, then from tins to 
calculate sin 3 y either from the formula 

sin 3y = sinj/(3 — 4sin 2 y) 

or by-use of inverse sine and sine tables, and then to evaluate 

f t (x) = 2 sin 3x -f- 3 sin 3y — 0- 3 

for these values of sin 3 y and the corresponding values of sin 3x. The work is eon 
veniently arranged in tubular form. 
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siny 

sin 3y 

sin 3x 

/a(*) 


= 4(1—sinar) 




0 

0-5 

+ 1 

0 

+ 2*70 

1(*7T) = 30° 

0-25 

0-688 

1 

3-76 

2(Jtt) = 60° 

0 067 

0-200 

0 

+ 0-30 

3Utt) = 90° 

0 

0 

— 1 

— 2-30 

4(^77) = 120° 

0 067 

0-200 

0 

+ 0-30 

5(^77) = 150° 

0-25 

0-688 

1 

3-76 

G(Jt7) = 180° 

0-5 

1 

0 

+ 2-70 

7(Jtt) = 210" 

0-75 

-f 0-562 

— 1 

-0-61 

8(Jt7) = 240" 

0-933 

-0-450 

0 

-1*65 

9(ir r) = 270" 

1 

- J 

1 

-1-30 

10(Jtt) = 300° 

0-933 

- 0-450 

0 

- 1-65 

1 1(Jtt) = 330° 

0-75 

+ 0-562 

-1 

-0-61 

12(*t7) = 360° 

0-5 

1 

0 

+ 2*70 


Two decimals are adequate to locate the roots approximately. A graph drawn from 
these values, or even inspection of the table without actually drawing a graph, 
shows that there arc roots in the neighbourhood of x/( Jtt) = 2*1, 3-9, 6-8, and 11*2. 

The approximate solutions so determined can be improved by tabula¬ 
tion at smaller intervals and inverse interpolation, or by an iterative 
process. If both the equations can be solved for one variable in terms 
of the other, say for y in terms of x: 

y — for the first equation. 

y = <f> 2 {x) for the second equation, 

then it may he more convenient to evaluate <j> 2 {x) as a function 

of x and interpolate for the zero of this function. 


Example: To find more exactly the root of 

sin .r-f 2sin// = 1, 2sin 3-r+3sin 3y = 0-3 

in the neighbourhood of u’/(Jtt) =2*1 


+ (4tt) * r ' 

sin ?/ 

4( 1 — sin .r) 

V = ^i(-r) ! 

sin 3 y y = <j> 2 (x) 

0-1 — § sin 3x 


2 0 

60 

-0670 

•0671 

•i 

•0334 

- -0337 

240 

2 05 

61 A ’ 

•0606 

•0607 

• 1523 

•0510 

- 0097 -2 

238 

2 1 

63° 

•0545 

•0545 

•2043 

.0686 

+ 0141 -5 

233 

2 15 

64 A’ 

•0487 

•0487 

•2556 

•0862 

+ 0374 -2 

231 


66 

•0432 

•0432 | 

i -3060 

•1037 

+-0605 

and 

J' 

inverse int 
2140 (Jt7). 

erpt ilnt ion 

then gives 

the required 

solut ion. 

approximately 
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When neither of the equations can be solved formally for x or y, 
the same processes can be used, one or both of the functions <f> x {x), <f> 2 {x) 
being determined roughly graphically, or more accurately numerically, 
by solution of the equation f x (x, y) = 0 or f 2 (x } y) = 0 for y in terms of x. 
For example, if a set of graphs of f x (x, y) against y for a set of constant 
values of x is constructed, the intersections of these graphs with the 
t/-axis give the function y = <f> x (x) which can then be used to substitute 

for y in the second equation. 

Another process is to evaluate f x (x,y) and f 2 {x,y) for a set of points 
on a coarse grid in the {x, y) plane, and on a piece of squared paper to 
mark at each (x,y) point the values of f x (x,y) and f,{x,y) there. The 
loci f x {x, y) = 0 and f 2 {x, y) = 0 can then be sketched roughly, and the 
intersections of the curves thus sketched then indicate the regions o 
the plane in which a closer examination is necessary in order to determine 

the roots more accurately. 


(9.13) 

(9.14) 


Example : To locate approximately the real solutions of 

xy(2x*-y 2 )+mx + y) = 48 » 

X 2 + »/ 2 = 16. 

The second equation shows that x and y lie between ±4, so evaluate 

xy(2x 2 -y 2 )+\G(x + y) 

on a square grid of mesh side unity in the (x,y) plane for |x| 4, |./| ( 

Fig. 16). Although this grid is a coarse one, it enables the contour/,(x, y) 

is, ar»/(2x 2 —y J )+16(x+i/) = 48, 

to be sketched roughly. In this particular case the contour U^.y) =°* th«vt i*. 
*>+»* = 10. could be drawn accurately; but in the figure .t baa been sketched 
freehand from the values of x‘ + y* at the mesh points, as would have to bo don 

“ ThTtatoiscctions of the two contours show that there are four real solutions, 
approximately: ^ _ _ g _ Q ^ = ^ * = j.g * = 4 0 

y = 3-5 y = 40 y — 3 - 6 y = —02 

and probably two in the neighbourhood of x - -3-5, „ = 

tion of function values on a finer grid would be necessary in order to make certain 

whether the contours intersect in this region. , / (r 

Notes: (i) The values recorded in the figure arc not those of/»<*.*> U *- /) 
themselves, but are those of the left-hand sides of equations ( 9 - 1 3).(9-l ; 

(ii) Sinco/,(x, y) in this case is a cubic in x for fixed y and a cubic in yto i *«*£ 
the table of its values can be checked both very easily and xery thor g 
differencing in both directions. Alternatively, these values coult^ 
the differences in the y direction and checked by differencing in the x 

(iii) Since this method involves evaluating the function on a ^ 

points, it should in general be avoided when it is possible formally to use ether 

F 


6353 
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1 \> ,\> 17 /6 17 20 25 32 

■ ■ . I 'iii 


- xy(2x*-y : ) + 16 (x + y) = 48 

. x 2 + y 2 = 16 

Fig. 1G 


°f ,llp equations to solve for one or otlier variable in terms of the other, and so to 
reduce the problem to one in u single variable. It is used in this example to pro- 
vide the possibility of comparing this method, and the results of using it and of 
improving the solutions, with others. 

A convenient alternative method for these equations is to use the substitution 
teostf. »/ »sin0, which ensures that the equation (9.14) is satisfied, and 
tl.cn to treat equation (9.13) ns an equation in 6. Another alternative is to uso 
equation (9.14) to substitute for//- in (9.13), which then becomes 

f(3.r= 10 ).r t- 1 (? J y 4 S - 1 f>.r, (9.15) 

an I then treat . quatinns (9.14) and (9.15) as two equations of the form y = <f>(. r). 

(i I T lie be>t method of improving the approximate solutions will depend on 
the equal ion and may be different for the different solutions. In the present 
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example an iterative process is most convenient for the solutions near x - 0-4, 
y = 4 0,and x = 4 0, y = -0-2. Consider the latter. We use equations (9.14) and 
(9.15) alternately, the first to determine x from an approximate value of y, and the 
second to determine y from an approximate value of x. Since |y/x| is small for this 
solution, the value of x obtained from (9.14) is insensitive to the value of y taken 
and is not much altered when an improved approximation to y, derived from (9.1 o) 
with this value of x, is used. The iterative process, though only first order, con- 


A similar treatment, using an equation obtained by substituting for^r 2 instead 
of for y 2 in (9.13), is similarly effective for the solution near x = 0-4, y - 4-U. 


The following is a general process for improving the approximate 
values of a solution, when neither equation can be solved formally for 
either variable in terms of the other. The functions ,y) 

are evaluated on a finer grid of points (.r, y) in the neighbourhood o t e 
solution. But instead of the function values being recorded in the (x } y) 
plane, the value of / 2 (z,y) is plotted against /,<z,y) for each pair of 
values (z,y), and curves of constant * (z-contours) and of constant y 
(y- contours) are drawn in the (/„/,) plane. The advantage of this 
method of representing the behaviour of the two functions of z am y is 
that each curve iB drawn through plotted points instead of being inter¬ 
polated ‘by eye’ among an array of function values. 

Such a plot of U against /, is made for such values of z and y that the 

point /. = /. = Ois enclosed between two z-contours and two y-contours. 

Within a small enough region not containing more than one solution 
the z-contours and y-contours will usually be nearly equally spaced and 
not very curved, and if this is the case, it is possible to estimate fairly 

closely what contours pass through the point /, f 2 — 0. ca cu a ‘ 10 
of (/„/,) for this approximation to the solution then suggests for what 

further values of (z, y) the function should be evaluated in order to enclose 
the point /, = /, = 0 still more closely. The process is illustrated by 

the following example. 


Example: To find more accurately the solution of the equations 

*y(2x 2 -y 2 )+16(x + y) = 48, x 2 + y 2 = 19. 


in the neighbourhood of x = 1-8, y — 3-6. , 

From Fig. 16 it is estimated that the solution lies between x ’ t 

between y = 3 5 and 3-7. The improvement of this solution can bo carried out 

by the following procedure. A set of values of / x ant f 2 is irs c va enough 

3 y 0, and 37. z - Ml to 20. this range of* being taken m order 

values to check by differences. The x-contours an !/^ on “ r enclose the point 
points are shown in Fig. 17; those for y = 3-5 and 3 0 already r ^ ^ 

/, =/, = U; but some points for y = 3-7 have been calculated ch^k the spae ng 

of the y-contours and to show the curvature of the x-contours. lho (/,./.) 
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for x -- 1 -85, y — 3-55 is also shown. The solution estimated from these contours 
was a: = 1-84, y — 3-55. 



Values of (/,./ 2 ) are now calculated for these four sets of values x = 1-83, 1-84 
and (/ ^ 3-55, 3-5(5, anil the results plotted on a larger scale. For this small range 
of .r and y, the contours can be token us straight and equally spaced to tho accuracy 
of t lie plot. Inverse interpolation in x and y is required to determine the values of 
x and y to give/, = f, = 0, and this is most easily done by measurement. Tho 
values obtained can bo checked by calculating/, and f 2 for them. 

.\utcs: (i) As in Fig. 1(5, the functions plotted in Fig. 17 are the left-hand sides 
of the two equations, namely/,(x f y) f 48 und f 2 (x,y) + 16. 

(li) A convenient way of carrying out the final interpolation is as follows. Con¬ 
sider the interpolation between the two x-contours, say x = x 0 and x,, and let 
•< •«'„ .r 0 ) be the interpolated value required. Lay a ruler on the (/,,/.,) 

diagram so that its edge passes through the point/, =/, = 0; rotate it about this 
pu.nt and move it in the direction of its length until it cuts both x-contours at 
exact graduations on the scale, at a convenient interval (say 5, 10, or 20 units 
of t he scale graduation); then the value of the fraction 0 of the x-interval between 
the contours can be read off directly. 

(id) Tins method of plotting is not satisfactory for tho preliminary location of 
roots, since (x,y) is not in general a single-valued function of (/„/,), so that two 
or more x contours and two or more y-contours may pass through each point in 
a legion of the (/,./,) piano. If this occurs, and it will occur if the equations have 
more t han one solution, the (/,./ 3 ) diagram becomes complicated and its inter- 
|»i»(a! inn needs considerable care. 


‘>.7. Three or more variables 

1 here I- HO sat ..factory practical method, graphical or tabular, of displaying the 
'• l.av.om ol fi.net ions of three or more variables, ami the approximate location 
solid inns of such equations is therefore difficult-. 
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In some cases the solutions can be regarded as representing the asymptotic 
steady-state values of the solutions of a system of differential equations, and then 
they may be determined approximately by integrating this system of differential 
equations. If the equations arise from some scientific problem, this may suggest 
the appropriate differential equations to use. For example, in the chemical equili¬ 
brium of a system of a number of gaseous components, the relations between the 
concentrations of the components are given by a set of non-linear equations in¬ 
volving the equilibrium constants of the various reactions. If, for example, one 
of the reactions was 2C0 + 0, ^ 2C0. and tf(X) stands for the concentration of 
the molecular species X, one equation would bo 

[tf(CO)] 8 W,) = K£C( CO,)] 2 . (9.16) 

But the equilibrium is attained through a non-steady process in which the con- 
centrations of the components change with time, that of oxygen, for example, being 
given by an equation 

0 2 ) = ptlKMC O s )}*-*(<W(CO)} 4 ], (9.17) 

(It 

and similarly for other components. We can try to make the calculutions approach 
a steady state by following out such a time-varying process. However, since 
the purpose of the differential equation (9.17) is solely to provide a means for 
approaching a solution of the equation (9.16), there is no need to take experimental 
values of the reaction rate coefficients like 0, in (9.17) even if these are known; 
an artificial set can be taken, convenient for the numerical work, and they need 
not oven be tuken to be constant. 


X 

FUNCTIONS OF TWO OR MORE VARIABLES 


10.1. Functions of a complex variable and functions of two 
variables 

There are two rather distinct contexts in which functions of two or 
more variables may arise in numerical work. One is concerned with 
complex numbers and functions of a complex variable. In numerical 
work it is usually best to treat a complex number as a pair of real numbers, 
either ( x,y ) in the Cartesian form (2 = x+iy) or (r, 6) in the polar form 
(2 = re i0 ) of the complex number as is most convenient for the calcula¬ 
tion concerned. In this context a particularly important feature is the 
property of any analytical function f(z) = g(z)-\-ih(z) of a complex 
variable 2 , that its real and imaginary parts both satisfy Laplace’s 
equation in two dimensions. For this reason, the finite difference form 
of the two-dimensional Laplacian operator (d 2 /dx 2 ) + (d 2 /dy 2 ) plays a 
particularly important part in such contexts. 

The other is the general case of funct ions of two or more real variables 
other than those arising from formal expressions involving complex 
numbers. Here, too, the finite difference form of the Laplacian operator 
is important, particularly in two dimensions, and in three dimensions 
with some degree of spatial symmetry. 


10.11. Numerical calculations with complex numbers 

The details of numerical calculations with complex numbers will be 

carried out almost entirely with pairs of real numbers, since there is no 

standard calculating machine which deals directly with complex numbers. 

For addition and subtraction the Cartesian form 2 = x-\-iy is clearly 

the more convenient. For multiplication and division the polar form 

2 = Isle'* seems preferable to the Cartesian form since although the 
Cartesian formulae 


( r i i ? !/i)(- r 2+ * 1 / 2 ) — X l X 2 yi2f2~\~^( r iV2^~ x 2!/i)> 

(* r ! + *yi)i( x 2+ hv 2 ) = |>1 x 2+!/i 2/2-1- *(— *1 y 2 -f .r„ y,)]/(a-|-f y\) 
arc not difficult to evaluate, it is also not difficult to make a mistake of 
sign in this evaluation, particularly when x lt x 2 .y lt and y 2 are not all 
positive. I sc of the polar form will,probably involve some conversion 
lroni Cartesian to polar form; various good modern books of tablest 


f For example*, Chambers'. ■> G-Figure Tables, vol. 2 (1949). 
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include tables for simplifying this conversion. Whether, and at what 
stages of a calculation, it is advisable to make a conversion from Cartesian 
to polar form or vice versa will depend so much on the calculation, and 
also perhaps on the individual worker and on whether Cartesian-polar 
conversion tables are available, that no general rule can be laid down. 

For finding powers (other than squares and perhaps fourth powers) 
or roots of complex numbers, the polar form is usually the most con¬ 
venient. But square roots can be found directly from the Cartesian 

form as follows. Let (x-j-iy)* = 

where f, v are real. On squaring and separating real and imaginary 
parts this gives I 2 —t/ 2 = x > 2 €v = V- ( 10 - 1 ) 

Elimination of rj gives a quadratic for £ 2 , of which only the positive 
root is significant since £ is real; this root is 

Whence | = [«.+,(*+*■), Iff. 00.2) 

the signs of these square roots being taken so that 2£r) = y. If * 19 
positive it may be best to use the first of formulae (10.2) to determine 
and then to find v from v = ?//2£; and similarly if * is negative to use 
the second of formulae (10.2) to determine rj, and then find t from 
£ _ yl2rj. The result can be checked by squaring the value of (f + iT ?) 

obtained. 

10.2. Finite differences in two dimensions; square grid 

Just as for functions of one variable * we often have to consider func¬ 
tions as specified by a table at discrete values of *. usually at equal 
intervals, so for a function of two or more independent variables Me are 
often eoneerned with a function specified at discrete, equally spaced, 
values of all the independent variables. In particular, with two inde¬ 
pendent variables (x,y) it is very often most convenient to take these 
discrete values of * and y in such a way that they form a grid of square 

mesh in the ( x , y) plane, such as 

{x,y) = {x 0 +jSx,y 0 +kSy); to = ty = < I0 - 3 > 

with integral values of (j, k). The values of a function / at such a point 

will be written f, k . . .. ,, 

Such a function can be differenced in the * direction and in the j 
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Sy = 2 sinh £ U v , 


(10.5) 


direction; S x , 8 y will be used for central difference operators in the x and 
y directions, so that 

&xfj,k — $ufjjc = i i 

^xfjjc ~ fj+l,k— 2 fj.k+fj-l.k> &lfj,k = fj.k+\~^fjjc+fijc-l‘ 

A particularly important relation is 

= fj+i.k+fj,k+i+fj-i,k+fj,k-i—4fj,k- (10.4) 
The operators 8x(8jdx) and 8y(d/dy) will be written U x , U y , the notation 
being an obvious extension of that of §4.7. Then, as in §4.7, 

8 X = 2 sinh W x , 

and 8-+ 8“ = 2(cosh {7 x + cos h U y )—4. 

It is convenient to represent formulae such as (10.4), which represent 

linear combinations of values of / at a set of 
neighbouring points in the (x,y) plane, in a 
diagrammatic form in which the way in which 
the different function values enter is more 
immediately evident. Bickleyf uses diagrams 
in which the set of coefficients in formula 
(10.4) would be represented by Fig. 18. A. 
similar diagram, however, is used by Southwell 
in a different sense (see § 10.61, Fig. 19), and 
its use to represent the coefficients in formula 
(10.4) might be confusing. A more convenient form for printing is the 
following diagrammatic representation of formula (10.4): 



<« 2 +«})/** = 


0 

1 

0 

1 

— 4 

1 

0 

1 

0 


fiJc* 


the set of coefficients being enclosed in a ‘box’ to distinguish it from a 
matrix. Another formula which will be needed, and which can be 
written in a similar form, is 





1 -2 1 

1 -2 

I 

*!/« = 

1 

tv 

** 

1 

to 



1 -2 1 


fj,k- (10.6) 


llw- (juantitv (5“4 8-)fj k is four times the difference between the 
arithmet ic mean of the values of/at the corners of a square centred on 
t he point ( j. k) and the value of / at the centre, the corners of the square 

t ^ Biekley, Quart . J. Merit, ami Applied Math . 1 (1948), 35. 
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being the grid points which are the nearest neighbours of (j,k) (the side 
of the square is V2(8s), not 8s). A similar quantity involving next- 
nearest neighbours is 



In terms of the operators U x , U y this is 
[e^**c/ r _|_ e -i/,+ u,_i_ e -u x -u,_^ e u x -u t — 4]fj k 

= [2 cosh( U x + U v ) + 2 cosh( l r x — U v )~ 4]f Jk 
= 4[cosh U x cosh U v — 1 
In terms of the differences of / it can be written: 



10.3. The operator d 2 /dx 2 + £ 2 /cty 2 

The particular importance of the Laplacian operator in two dimensions 

has already been noted in § 10.1. On a grid of square mesh of side 8s 
we have (8 s) 2 (d 2 /dx 2 -\-d 2 ldy 2 ) = U*+U 2 vt 

and arc therefore concerned with finite-difference approximations to 
in + U 2 . Using the approximations of §4.7 we can express this in 

terms of the operators 8 2 and 8“ as follows: 

U 2 + Vl = 8|[(sinh -1 iS jr )/iS,] 2 -4-S*[(sinh~ l tt v )l& v Y 

= SJ-tfS+0(8») 6 +8J - tft+0(8y )• 

= SJ+S}-A(^+8t)+°(S«) # * (108) 

Thus the simplest approximation to U 2 -\r UJ is 

U 2 + U 2 = 8 2 -^8 2 +0(8s)\ 



which gives 


( 10 . 9 ) 
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with an error term of order (8s) 2 . This approximation is widely used 
in numerical work. In particular it gives 


0 

1 

0 

1 

— 4 

1 

0 

1 

0 


fi* = 0 


( 10 . 10 ) 


as a finite-difference form of d 2 fjdx 2 -\-d 2 fldy 2 = 0. 


10.31. Special relations when d 2 fldx 2 -\~d 2 f/8y 2 = 0 
In many contexts in which the operator {d 2 jdx 2 -\-b 2 jdy 2 ) arises, its 
importance comes from the fact that one or more of the functions f(x,y) 
concerned satisfy the relation 

3*flda*+d*fjdy* = 0. 

This is always the case when we are concerned with analytic functions 
of a complex variable, and is often the case in calculations not directly 
concerned with complex variables. If the operands are restricted to 
such functions, we have T:2 , r2 , 

x I V 

and this can be used to obtain some special formulae for use in such con¬ 
texts; but it must be remembered that they are restricted to such 
operands. 

One of the most important can be derived as follows: 

Since l' 2 f l i = 0 it follows that 


ui = v\ = - v% u*. 

Hence = -2S*S» + <W, 

so that lormula (10.S) can be written 

o = S 2 -f 5 2 -f-J8 2 8 2 -f-0(85) 6 , 
and substitution from (10.7) gives 


( 10 . 11 ) 


that is, 



1 

0 

1 

4(82+sS>/m+ 

0 

-4 

0 


1 

0 

1 


i 

4 

1 



4 — 

20 

4 

fj.k = 


1 

4 

1 ! 



fj,k = 0( Ss)« 


( 10 . 12 ) 


\\ it n an eiror term of order (S.s)' 1 . 1’lxis is an improvement on the simplest 
finite difference form (10.10) of the equation ?'~f/dx 2 +8 2 f/dy 2 = 0, for 
which the error is of order (8s) 4 . 
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Another consequence of the relations (10.8) and (10.11) is that 

SJ = 6(8i+s;)+0(8s) 6 , (10.13) 

and this can, if Convenient, be used in integration or interpolation 
formulae to substitute for fourth differences in the z direction in terms 
of the second differences. For example, one formula for integration 

in the x direction is 

x 0 + 8* 

j fd X = 2(&x)[f 0 +hVf 0 -^ , fo\ + OW (10.14) 

Xo —8x 

(this is equivalent to Simpson’s rule improved by the inclusion of the 
leading correcting term; see §0.3). Expressed in diagrammatic form 
in terms of function values, this is 




114 34 — 1 ]fijc + 0(Sx) 7 . (10.15) 


J /(*> Vk) dx = «o( S t ')[— 1 34 

a*-» 

Substitution for 8}/ from (10.13) in (10.14) gives 

x 0 + 8x 

f fix = 2(to)[/ 0 +- 1 ^/„- S , S 5 2 „/o] + 0(^)’, 

xo-8j 


r i* 1 

that is,t J f{x, y k ) dx = i&hx) 


XI-, 


o 

* 

0 

4 24 

4 

0 —1 

0 


A*+0( Sxy. (10.16) 


The coefficients are simpler in (10.16) than in (10.15) and the coefficient 
in the error term is smaller, as might be expected from the fact that the 
values of / involved in formula (10.16) lie nearer the range through 
which the integration is being carried than do the function values m 

(10.15)4 

10.4. Finite differences in cylindrical coordinates 

It is occasionally convenient to use finite differences at equal intci 

in polar coordinates (r,0) in a plane, or in cylindrical polar coori n , 

rather than in Cartesian coordinates. Plane polar coordinate, would 

be the natural ones to use, for example, in a calculation concerned with 

a solution of Laplace's equation in two dimensions with hounda^ 

conditions given on a circular boundary; and cy in nc.i ]>< a 

ordinates would be the natural ones to use in a three-dimensional problem 

t Thi. formula ... first derived by another method by G. BirkhofI end D. M. oung. 

POO Jo.tr,I. of Math, and Phjs. 29 (1050). t jnt lation of functions of a 

J l*or a niinilar uko of tho relation .. w f -v 37 (UMO), 230; 39 

complex variable. «oo P. M. and A. M. Woodward. Phtl. Mag. (.) 37 (U 

( 1948 ), 594 . 


220 


FUNCTIONS OF TWO OR MORE VARIABLES 


with axial symmetry and boundary conditions on the surface of a 
circular cylinder. These cases can be considered together, the case of 
plane polar coordinates being given by putting d/dz = 0 in the equations 
for cylindrical polar coordinates. 

One way of dealing with such calculations is to make the conformal 
transformation to (log r, 6) and to w'ork on a rectangular or square grid 
in the (log r,6) plane. But this is often not convenient w'hen the point 
(or axis) r = 0 is in the domain to be covered by the integration, and it 
is then better to use the (r, 0) coordinates without modification. 

Consider first the case of axial symmetry. Then the Laplacian operator 
in cylindrical polar coordinates is 





The finite-difference approximation to d^'dz 2 is the same as in Cartesian 
coordinates; only the r-derivatives need special treatment. Let / stand 
for/(j Sr). Then 

P). = (X + i-2/i+/-.)/(S<-) s +0(S<) 2 . 

An approximation to (t /jdr)j , with an error term of the same order, is 


so that for J 0 

'**/ , 1 <f 


i C £] = >/2(Si) + 0(Sr) 2 , 

V r /j 


cr 1 


r 'tr 




(Sr)2 + 0(Sr)* 


-ijfi (2/+ l)/ /41 ]/2j(Sr)2 f Oi&r)*. (10.17) 
for axial symmetry, either there is a singularity at r — 0 or (fjdr is 
zero there. If there is a singularity, further analytical investigation is 
required before numerical methods are applied. If dfjdr = 0 at r = 0, 
then j 

{Tr* +rh) 0 = 4 ^~ _ JoM Sr ) 2 +°( Sr ) 2 , (10.18) 

a relation which can also be obtained from (10.0), since for axial sym¬ 
metry each of the values of/ with coefficient unity in (10.9) is f x . 

It there is not axial symmetry, then there is an additional term 

>• - < " 2 in 1 lie Laplacian operator, and if f Jk stands for f(jSr,kSd), we 

have tor / / (), 

\ -= - * F‘ 1 fj.k i| 

r (jSr)-l (S»j= 


(10.19) 
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For the equation for j = 0, let J\ be the arithmetic mean of the values 
f l k of / on the circle r = 8 r. Then 

(W + lg+LfR = 4(/ 1 -/ 0 )/(8r)MO(5r)=. (10.20) 

\dr 2 r dr r l dd 2 / 0 


If f varies in the z direction, then to give V 2 /, a finite-difference 
approximation to 8 2 //dz 2 has to be added to whichever of formulae 
(10.17) to ( 10 . 20 ) is the appropriate one to use for the variations in the 

(r, 0) plane. 


10.5. Partial differential equations 

Solutions of partial differential equations can sometimes be obtained 
by a separation of variables, by which the partial differential equation 
is reduced to a number of separate ordinary equations, one in each of the 
independent variables. Such a separation, if possible, is part of the 
preliminary analytical treatment of the problem before numerical 
methods come to be applied, and will not be considered here. The follow ¬ 
ing sections are concerned with the numerical treatment of partial 
differential equations as such. It will mainly be concerned with partial 
differential equations in two independent variables, as the numerical 
solution of equations with three or more independent variables is 
usually a problem on too large a scale to handle without special equip 

ment. . 

Most partial differential equations which arise in contexts in which 

numerical solutions are required are second order in <it least one of t u 
independent variables, and, moreover, are linear in the second-order 
derivatives. Simple examples are Poisson’s equation in two dimensions 


3+S-x-* 


( 10 . 21 ) 


where g(x,y) is given; the equation of heat conduction or diffusion in 
one dimension 32 f (10 *>*>) 

dt = D dx 2 

in which the diffusivity D may depend on/(this would make the equation 
as a whole non-linear, but the second derivathe enters lint .11 \ ), 

the wave equation d ij ^zj (10.23) 

W = a dx 2 ‘ 

Just as the nature of the problem of numerical solution or ordinary 
differential equations depends on whether the conditions tu . 
has to satisfy are of the one point or two point type, so the nature ot the 
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problem of the numerical solution of partial differential equations 
depends on whether the boundary conditions are given on a boundary 
completely enclosing the domain of the variables over which a solution 
is required, or whether this domain is unbounded in one or more direc¬ 
tions. There is a classification of second-order equations in two variables 
as ‘elliptic’, ‘parabolic’, or ‘hyperbolic’ which is closely related to the 
different characters of boundary conditions usually associated with 
such equations, and the character of the problem of numerical integration 
is correspondingly different in the three cases. 

The general second-order equation in two variables, linear in the 
second derivatives, is 


H 


W + 2 k£L +L %L + m = o. 

ox “ oxey cy- 


(10.24) 


where II, K, L . M may be functions of any one or more of the variables 
x < /> Pfidx, <■'/ ?U- The classification depends on the sign of K 2 —HL; 
the reason lor this will be explained in § 10.8. If this quantity is negative, 
the equation is termed ‘elliptic’; if it is zero, the equation is termed 
parabolic’; and if it is positive, the equation is termed ‘hyperbolic’. 
Poisson’s equation ( 10 . 21 ) is a simple example of an ‘elliptic’ equation, 
the diffusion equation (10.22) is one of a ‘parabolic’ equation, and the 
wave equation (10.23) is one of a hyperbolic’ equation. ‘Elliptic’ 
equations are usually associated with a domain completely bounded by 
closed curves (one oi which may be the circle at infinity) on which 
boundary conditions are given. ‘Parabolic’ and ‘hyperbolic’ equations 
are usually associated with a domain which is open in the direction of 
one variable, which physically is often the time variable. For example 
we may require a solution of the heat conduction equation (10.22) from 
given initial conditions in time (/given as a function of x at t = 0) and 
with given terminal conditions in space (/ given as a function of t at 
x = a,x — />) but with no condition to be satisfied at a later time t — T\ 
the initial and terminal conditions are enough to define a solution, and 
such an independent condition at a later time could not generally be 
satisfied. It is not, however, necessary that the boundary conditions 
should be of this type: we might alternatively have no initial conditions, 
but given terminal conditions and a condition of periodicity in time, that 
is. a condition that / should be the same function of x at a given time T 
as at time t 0 . 

11 //. A, and L are not all constants, then the equation may be of 
different type in different parts of the domain in which the solution is 
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required. But in many of the simpler partial differential equations, 
such as (10.21), (10.22), and (10.23), including many practically important 
ones for which numerical work is likely to be needed, the equation 
remains of the same type throughout the whole domain, and only such 
cases will be considered here. 


10.6. Elliptic equations 

Poisson’s equation (10.21) in two dimensions will be taken as a typical 
example of an elliptic equation for whose solution we require a numerical 
process. This process will cover as special cases Laplace’s equation 
(y{x,y) = 0 in (10.21)) and the torsion equation (g(x,y) = const.). The 
first step is to replace the partial differential equation by a finite- 
difference relation on a convenient grid of discrete points. A Cartesian 
or polar grid will usually be most convenient, and for the present only 
a Cartesian grid of square mesh with mesh side h will be considered. It 
is clearly most convenient if the boundaries are of such a form that a 
grid can be chosen so that the boundaries lie along the sides or diagonals 
of the grid squares, and it will be supposed for the present that this is 

the case and that the grid is so chosen. 

Using the simplest approximation (10.9) to [(c 2 /dx 2 ) + {^ 2 /cij~)] we 
then have a set of equations 


(10.25) 


one for each mesh point. These are linear simultaneous algeb'aic 
equations, so that we have formally reduced the numerical problem to 
one of the kind already considered in Chapter VIII. r \ he solution of 
the set of equations (10.25) is not, of course, the solution of the partial 
differential equation on account of the truncation error of the approxima¬ 
tion (10.9). The approximation can be improved by taking a finer mesh 
or by using the better approximation (10.H) to d 2 f/d 2 x-\-£ 2 f/dy 2 . If the 
latter process is used, a convenient procedure is to write the finite- 

difference equation 





( 10 . 20 ) 


and to solve this by an iterative process, using in 


the nth stage of the 
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iterative process values of (8^-fSJ)/ obtained from the results of the 
(n— l)th stage, f 


10.61. Relaxation process 

A ‘relaxation’ process (§ 8.5) is very convenient for carrying out the 
numerical solution of the set of equations (10.25) or (10.26), and is 
commonly used for this purpose. This common association of the 
relaxation procedure with the approximate equations (10.25) seems to 
have given the impression that the relaxation process itself is approxi¬ 
mate, and the errors of the approximat ion (10.25) are sometimes referred 

% 

to as ‘errors of the relaxation process’. But this is a misunderstanding; 
the approximation is not in the relaxation process itself but in the 
equations (10.25) whose solution is evaluated by this process. Regarded 
as a solution of the partial differential equation, the solution of equations 
(10.25) is equally in error whether it is evaluated by the relaxation process 
or by any other (such as elimination or inversion of the matrix of the 
coefficients of equations (10.25)) and the errors have nothing to do with 
the relaxation process used to obtain a solution of these finite-difference 
equations. 

The approximation to the solution of the partial differential equations 
can be improved by reducing the mesh size of the grid on which the finite 
differences are taken. In practice it is advisable to start with a very 
coarse mesh so that the number of grid points is quite small, and then 
to break down the grid to one of smaller mesh size when an approximate 
solut ion on the coarse mesh has been reached. Then the relaxation process 
on the finer grid starts from a set of values which is already a fair approxi¬ 
mation to the solution. 

It is convenient to carry out the relaxation process on a diagram 
representing the domain in which the solution is required, with the finite- 
difference gr id drawn on it. The usual convention is to write the function 
values and their changes to the left of each grid point, and values of the 
residuals to the right, for the simplest finite-difference approximation 
(10.25) to Poisson’s equation, the residual R jk at the point (j,k) is 



0 

1 

0 

1 

— 4 

1 

0 

1 

0 




(10.27) 


If a relaxation A/ is made at one point, the residual at that point is 
changed bv —4 A/, and that at eacli nearest neighbour, other than a 


t Sec, for example, L. Fox, Proc. Hoy. Soc. A, 190 (1947), 31. 
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boundary point, is changed by -f- A/, so that the pattern of the changes 
in the residuals is as represented diagrammatically in Fig. 19. The 
entries here are the coefficients of a single A/ value. 


Example: To find approximately the solution of Laplace’s equation 

8 2 V 8 2 V 
dx 2 by 1 

for the system shown in Fig. 20, with equipotentials V = 0 and I' 
indicated. 


= 80 as 



Fio. 19. 


Fio. 20. 


The first, coarse, grid can be taken as shown in Fig. 21. It might seem at first 
sight that this grid is too coarse for the results to bo of any value. But wo shall 
see that this rough approximation is in fact useful, and is obtained much more 
easily and quickly than results on a finer grid. By symmetry, only half of the 
diagram need bo shown, but it must be remembered that each relaxation AI at 
a point one interval from the centre line is accompanied by an equal one at the 
image point, so that the contribution to the residual on the centre line is 2AI . 

A set of values of V from which to start the relaxation process can be written 
in as if the equipotential V = 0 were the straight line A D. Then the residuals are 
zero except on A D. These values of V and the residuals are entered on l- ig. 1 • 
We could start the relaxation process by making such a relaxation as to reduce 
the rcsiduul at C (for example) to zero; this would require a relaxation AK - +10 
at C (and at its image in the centre line). But clearly a positive relaxation A is 
going to be required at D, which will make a positive contribution to the residual 
at C, and a further positive relaxation AFatC will be needed to remove it. We 
can anticipate this by deliberately, taking a larger relaxation A1 at C than is 
required to reduce the residual there to zero; this is called over-relaxing . Expe¬ 
rience is the only way of learning when and by how much to over-re ax; the 
beginner will probably bo inclined not to ovor-relax enough. As a rough ru e 
may be suggested that when there are several residuals of the same sign togeth , 
over-relaxation by a factor 2 will not be excessive. 

Q 


6353 
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V=80 


Centre 

line 



Fio. 21 


Let us start, therefore, with a relaxation AF = +20 at C; this leaves a con¬ 
siderable positive residual at D, flanked now by large negative residuals at C and 
its image point, so t hat a smaller degree of over-relaxation is now required. Let us 
therefore take next a relaxation AT’ = + 25 at D. 

The further process of the calculation is indicated in Fig. 21, t ho relaxations and 
the values of the residuals resulting from them being indicated by (a), (6), (c),... 
in succession. At some convenient stage in the work, it is advisable to collect 
together t he values of A1 and write down a new set of F’s at the mesh points, and 
to recalculate the residuals from these values of V in case any mistake has been 
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made in the relaxation process. The calculation can then be continued from these 
values of V and the corresponding residuals. This check should normally be made 
before the accuracy of the numerical work is increased by taking an extra significant 
figure (compare the examples in §§ 8.51 and 8.53) and always before changing from 
a coarser to a finer grid. 

Notes: (i) The individual numerical steps of the relaxation process are very simple 
and are carried out with small numbers, usually of one or two significant figures; 
they can therefore be carried out rapidly and easily. 



Fig. 22. 


(ii) When the value of V at a point not at a distance 8s from a boundary is 
changed, the sum of the residuals remains unchanged; all that is changed is the 
distribution of this total among the grid points. But if a relaxation A V is made 
at a grid point P adjacent to a boundary, the sum of the residuals is reduced by 
AV, or by 2AV or 3AK if two or three of the nearest neighbours of r are on the 

boundary (see Fig. 22). , . .. . 

(iii) A physical analogue of the relaxation process, as applied to the finite- 

difference form of Laplace’s equation, can bo given by considering 1 oisson s 
equation for the potential of two-dimensional distribution of electrical charge, 


namely 


b'-V ci 2 V 
ftc* + dy* 



The finite-difference approximation (10.10) to the left-hand side gives 


0 

1 

0 

1 

— 4 

1 

0 

1 

0 


4t Th- Pjik , (10.28) 


and Wpij. is (to this approximation) the charge on a square oi biao « 
the point (j\k). Thus for any assigned set of values of V jk , the residuals R iik aie 
a measure of the charge distribution required to give the assigned potential 
distribution. The relaxation process can bo regarded as a process of slu ting t n.-> 
charge distribution about until it is ultimately all in the form of surface charge 
on conductors forming the given equipotential boundaries and none is left as space 

charge in the domain over which the integration is carried. 

The constancy of the sum of the residuals when a relaxation is made at a grid 
point not adjacent to a boundary corresponds to the constancy of the tota. space 
charge in the domain when some charge is taken from one grid point and disti .bated 
among its four nearest neighbours. The change in the sum of the residuals when 
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a relaxation is made at a point adjacent to a boundary corresponds to the transfer 
of some of the space charge to surface charge on the boundary. 

This analogy suggests that the aim of the relaxation process should be not only 
to make the residuals small but to make them not all of the same sign, so that their 
sum, represented by the total residual space charge in this analogy, is small. It will 
not in general be possible to reduce all residuals to zero in the least significant digital 
position; a sprinkling of values ± 1 with occasional values ±2 is the best that can 
be expected, and such a set of residuals, with mean value perhaps 0-1 or 0-2, 
probably indicates a better approximation to a solution than a set of residuals -f I 
over the whole field. 

(iv) A set of residuals of magnitude not greater than 2 does not necessarily mean 
that the values of V are correct to a unit. It is advisable to reduce the residuals * 
on the final grid to ± 2 in the next figure beyond the last figure in V required in 
the final results. 

(v) No indications such as the letters (a), (6), (c),... in Fig. 21 are required in 
actual working; they are only given in this figure to help the reader to follow the 
details of the calculation. As soon as one value of a residual is replaced by another, 
the earlier one can bo crossed out or erased as being of no further interest. 

(vi) If the grid is drawn in ink and the working Ls done lightly in pencil, then old 
values of V and old residuals can bo erased without losing the pattern of the grid. 
This erasing of old values need not be done at every relaxation, but only when the 
space for values of residuals gets filled up. 

(vii) In this example the over-relaxation by a factor of 2 in the first relaxation 
has been a little too much, and a small relaxation of the opposite sign has had to 
be made later. But this step of over-relaxation has speeded the approach to a 
solution of the finite-difference equat ion. Only fourteen steps of relaxation have 
been needed to reduce the greatest value of | R jk | from 40 to 2. 

(viii) With the very coarse grid used here, there is no point in trying to improve 
the approximation to the solution of tho finite-difference equations by taking an 
extra figure in the T’-values. The next step is to reduce the truncation errors by 
taking a finer grid. 


10.62. Reducing the mesh size 

At some stage in the calculation it will usually be necessary, as in 
the above example, to change from a coarse to a finer grid. Let h x be 
the mesh size of the coarser grid. A convenient first step is to take the 
diagonals of the squares of the old grid as forming a new grid of mesh 
size h 2 = h v \2 (see Fig. 23). The new grid points are the centres of the 
squares of the old grid. For Poisson’s equation we have on the new grid, 
w ith the finite-difference approximation adopted 




ffo i o 

1 0 1 

l o 1 0 





and hence 


j 

4 


(10.29) 
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Fig. 23. 


This gives a set of values of/ at the centres of the squares of the old grid, 
which are the new grid points of the new grid. In particular for Laplace’s 
equation we have, in this approximation 



(10.30) 


that is to say, the value of / at the centre of a square is the arithmetic 
mean of its values at the corners. A further relaxation may be carried 
out on this grid, or this process may be repeated immediately, giving a 
grid of mesh size h 3 =‘\h x , whose grid points are the corners, centres, 
and mid-points of the sides of the original grid (see l’ig. -1). 


Example: The example of the previous section continued. Fig. 25 thews the 
process of breaking down the grid in two stages. Iho numbers in squares art 
values of V obtained in the calculation shown in Fig. 21. The numbers in circ es, 
at the centres of the squares of the original grid, are obtained by the npp ica ><>'> 
of formula (10.30) to the intermediate grid formed by the diagonals of t k* <>i 'g‘'> )l 
grid. The numbers at the other grid points of Fig. 25 are then obtain. < > '<> 

application of formula (10.30) to tho grid formed by the diagonals of the in c - 

mediate grid. , , , 

The residuals are shown on the right of the grid points, and only a single st. j f 

relaxation is then required to reduce the greatest \R»\ to 2. At tins stage another 

significant figure can bo taken in V and the relaxation process continued. 
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Note: The advantage of starting with a very coarse grid will now be apparent. The 
number of grid points varies as I fhr and the number of relaxations at each grid 
point probably varies roughly as 1 /h, so that if the finer grid of Fig. 25 had been 
used from the beginning, something like eight times as much work would be 
required to reach the stage represented by the results in Fig. 25. In terms of the 
analogy explained in note (iii) of the previous section, despite the coarse grid of 
Fig. 21, relaxation on this grid has carried out the bulk of the transfer of charge 
from the inter-electrode space to the electrodes, and what has to be done on the 
finer grid is mainly a minor rearrangement of the residual charges. 
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Fig. 24. 

10.63. Further notes on the relaxation process 

We have only been concerned here with the simplest case in which (i) the 
boundary of the domain of integration does not cut the side of any 
of the grid squares, (ii) the boundary condition is that V is given, and 
(iii) the equation to be solved is the simplest example of an elliptic equa¬ 
tion. For extensions of the procedure to deal with boundaries which 
cut the sides of some of the grid squares, with boundary conditions 
involving the normal derivative of V, and with less simple equations, 
for further practical hints on carrying out the relaxation process in this 
context, and for examples, reference should be made to Southwell’s 
Relaxation Methods in Theoretical Physics and papers referred to in 
the bibliography in that book.f 


t See also E. Stiefol, Zeit. f. angew. Math, und Phys. 3 (1952), 1. 
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Fio. 25. 


The method can he adapted to characteristic value problem, such 
as the determination of the values-or at least the smallest v alue-^f 

A for which the equation 

Sx^'cnf 

with boundary conditions / = 0 on a closed curve, has a non-trivial 
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solution. Use of the finite-difference approximation (10.9) on the left- 
hand side gives the set of simultaneous equations 


0 1 0 
1 -(4 ~Xh 2 ) 1 

0 1 0 



and the determination of A by a relaxation process follows the general 
lines of § 8.7. 


10.64. Richardson-Liebmann process for Laplace’s equation 

There is another process of successive approximation for solving the 
set of equations (10.10) which form the simplest finite-difference approxi¬ 
mation to 

S-S_o 

ex^dif~ 

In its simplest form, given by Liebmann.f this process consists of re¬ 
peated use of formula (10.30), working systematically over the grid, 
replacing/at each grid point by the arithmetic mean of the value of/ 
at its four nearest neighbours. 

In another form, given by L. F. Richardson,J each value oif jk in a 
trial solution is increased by a multiple a of the residual R jk at that 
P'-int, and the result is taken as the next trial solution. Richardson 
proposed the use of a set of different values of a in the construction of 
successive trial solutions. Liebmann’s process is equivalent to a special 
case of Richardson’s in which a is kept fixed. 

1 om pared with the more recently devised relaxation process, the 
Richardson Liebmann process has three disadvantages. First, all the 
work is done with large numbers, the values of / themselves, whereas in 
t he relaxation process the bulk of the work is done with relatively small 
and simple numbers, the relaxations of/and the residuals. Secondly, 
a lot of time and work is spent on calculation in regions where the 
residuals are small, whereas in the relaxation process attention is first 
directed to the region where the residuals are large and the rest of the 
domain is left untouched until the larger residuals have been removed. 
And, thirdly, it is not so easy to modify so as to take into account the 
higher differences in the replacement of derivatives by finite differences. 

10.7. Parabolic equations 

Processes for the numerical solution of parabolic and hyperbolic 
equations have been much less explored than those for elliptic equations, 

t H. I.iol'inann, SitzutujslM-r. Buyer. .1 lead. Munchcn (1918), 385. 

+ L F. Rirhurdson. Bhil. Trans. Buy. Soc. A, 210 (1010), 307. 
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though several practicable methods have been devised; some of these 
are indicated in the following sections (§§ 10.71-10.73). 

As a simple case of a parabolic equation we will consider the equation 
of heat conduction in one dimension 

d l — (10.31) 

dt ~ dx 2 ’ 


with given initial and terminal conditions. 

One way of dealing with this equation is first to replace only one of 
the derivatives by a finite difference; this replaces the partial differential 
equation by a set of ordinary equations which can then be treated by one 
of the methods of Chapter VII. The form of this set of ordinary equations 
and the process for their solution differ considerably according as it 
is the first-order (time) derivative or the second-order (space) derivative 

which is replaced by a finite difference. 

10.71. Replacement of the second-order (space) derivative by a 
finite difference 

Let fAt) be written for the value of/at .r =jhx and at time t. Ihen 
replacement of the second derivative by a finite difference gives 

(10.32) 

dt 

This is a set of simultaneous first-order equations for the different func¬ 
tions /,(() and these can be solved numerically without difficulty. The 
initial values of each/, is given by the initial conditions. The truncation 
errors are of order (8a;) 2 ; they can be estimated, and the leading term 
in the corrections applied, by Richardson's /^-extrapolation process 
(see § 7.51). This method is not restricted to one space variable and the 
time variable, and it is practicable to use it for the numerical solution 
of the equation of heat conduction in two space variables and, moieox cr 
for a substance of which the thermal propert ies vary with temperature.'t 


10.72. Replacement of the first-order (time) derivative by a 

finite difference . 

For a time interval 8/, the time derivative at any value of * can be 

replaced by a finite difference as follows: 

= [/(x,f+S0-/(^0]/( 8 0 + ^( 8 0 2 » 

and, with an crrlr term of the same order, 8*//ta* at time t+ 18 1 can be 
f Soo N. B. Eyros nn.l other*. PM. Tram. Ito.j. Soc. 240 (1910), 1. 
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replaced by the arithmetic mean of its values at the beginning and end 
of the time interval: 

@L +l „=^[S {/(i - ,+8 ° +/(a: ' e)} ] +0(5<)2 ' 

If the right-hand sides of these are equated and the error terms neglected, 
i we have 


d 2 

dx 2 


[f(x,t+8t)+f(x,t)] = (2/80[/(^<+80+/(x,0]-(4/80/(^0 


(10.33) 

Given/ as a function of x at time l, this is an ordinary differential equation 
for / as a function of x at time /-f St. There is a set of equations (10.33), 
one for each time interval. But they can be integrated successively, 
and do not have to be treated simultaneously as equations (10.32) do; 
the calculation proceeds interval by interval in t, the results f(x ,/+£/) 
for the end of one interval being the given function f(x,t) for the begin¬ 
ning of the next. 

In the integration of equation (10.33) it is not necessary to know the 
values of d 2 f\cx 2 at the beginning of the interval; the best procedure is 
to carry out the numerical solution regarding equation (10.33) as an 
equat ion for \f{x, t-\-St)-\-f(x, /)] and then to subtract the known 
f{x,l) to give f(x, t + St). 

If two separate integrations covering the same range in t are carried 
out, with different time intervals St, the leading term in the truncation 
error can be eliminated by Richardson’s /^-extrapolation process (see 
$ 7.51). and in many cases this will also correct for the next term in the 
truncation error.f 

Although in this method we carry out successive integrations of a 
single equation (10.33) instead of simultaneous integrations of a set of 
equations, and, moreover, equation (10.33) is a second-order equation 
with the first, derivative absent, which as mentioned in § 7.2 is the most 
convenient form of all for numerical treatment, the method has the 
gieat disad\antage that the solution of this equation has to satisfy two- 
point boundary conditions in .r, and a step-by-step integration is often 
difficult because of the extreme sensitiveness of the solution to the 
initial conditions and to rounding errors. This sensitiveness is more 
marked the smaller the value taken for the time interval St, so that 
while a small value of St would be preferred in order to keep down the 
truncation errors, it may make the integration process impracticable. 

t Soo D. It. Hartr.» mvl .1. It. Womcrsloy. I'roc. Rn<,. Soc. A, 161 (1937), 303. 
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The two-point character of the boundary conditions and the sensitive¬ 
ness of the solution makes the method less simple and straightforward 
than it may appear at first sight, and in its application to less simple 
equations other precautions may be necessary in using the finite- 
difference approximations to derivatives.'f 


10.73. Replacement of both derivatives by finite differences 
In the notation of § 10.71 


(%) = [/,«+ 80 -/,«- 80 ]/ 2 S<+O( 80 * 
and (gj = [/ i+ 1 ( 0 - 2 A( 0 +/i-i( 0 ]/( 8 *)*+O(S*)*. 

These approximations invite us to equate the right-hand sides and so 
obtain (neglecting the error terms) 

8 <) = /,(<—80—{28</(&r) 2 }[/ i+1 (0— 2f j (t)+f j . i (t)]- 

This looks a very attractive formula, since if the solution has been 
carried to any value of t, it gives directly each value of fj(t-\-St) separately 
in terms of known quantities, and a process of using this formula to 
integrate through successive intervals St looks simple and stra 'o 
forward. Unfortunately, however, such a process is unstable, and effects 
of rounding errors build up rapidly and uncontrollably. J 

However, there is another way of using similar approximations w ic 
leads to a stable numerical process which is practicable but not quite 
so simple.:£ This is based on equating approximations to dfjdt and 
d 2 f/dx 2 not at grid points in the (x, 1) plane but at points ha wa\ 

t between grid points. We have 


= [/j((+ S,)_/,<O]/8*+0(SO 2 . 

W)x,MU 2 [Wi^u * MJ 

= {l/2(8jr) 2 }[/ J+1 (<+8()-2/ i (( + 80+/ji(' + 80 + 

+/, +1 (0-2/,(0+//-.(0] + O(S') 2 +O(8r)=. 

If we equate the right-hand sides of these expressions and neglect (lie 
error terms, we have 

/ J+1 (i+80-2{l+(8ar) 2 /2(S0i/;«+80+X-.(H-80 

= -[/ /+1 (0-2(l-(8*) s /28<}//(0+//-i«)]- < 1<U4) 

t For examples an,l further dtow.ion.roo P. R. Horlroo, Itcp. a„J Men. A.K.C.. 

No*. 2420. 2427 (1939). r l0 

t Hoo J. Crank and P. Nicolson, Proc. Comb. Pint- Soc. 43 <1° >■ 1 
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This is a set of simultaneous algebraic equations for fj{t+St) as a function 
of Xj with boundary conditions of the two-point type in Xj\ they can 
be solved by an application of the relaxation process or by some other 
process of successive approximation. 


10.74. Note on methods for parabolic equations 
In all three of the methods considered in §§ 10.71 to 10.73 the pro¬ 
cess of evaluating an approximate solution is carried out in the 
direction of t increasing, t being in the conduction equation (10.31) the 
time variable, and in general that independent variable which does not 
, occur in any second derivatives. All three methods are practicable 
onty if the domain of integration is open in the direction of this variable, 
so that the whole solution does not have to satisfy any conditions at some 
later time in the course of the process of solution. As already mentioned 
in § 10.5 this is the most common situation with parabolic equations. 


V 

1 


10 .8. Hyperbolic equations. Characteristics 

For hyperbolic equations methods similar to those for parabolic equations can 

bo used, and in addition there is another class of 
methods peculiar to hyperbolic equations. These 
depend on the properties of sets of curves called 
‘characteristics’ of a hyperbolic equation. As in 
§ 10.5, let the equation bo 

H S + 2A 'J% +£ ^ + j,/ = 0 ’ 00.35) 

w here H, K, L, and iV/ may be functions of any one 
or more of x, y, f, df/dx, df/dy, and consider the in¬ 
tegration of the equation along a curve C in the 
(x.y) plane. Let dy/dx be the gradient of C at (x,y) 



(sec 1* ig. 26). Then for an clement of arc ds of C 

dm = m -4 d Jl\ , fv J?f_\ ( &f dx . d-f dy\ , 

Vx. \cx* ds dxdy ds) ’ d \0y) \dxty ds+ey'-fcl d9 ’ 

and hence 

// - / . v d(^) p l (l f- d(^) = \(// c 2 *1 + L m dx d v , l n ( d yY -. , /d*\*n . 

°n substitutiun from the differential equation (10.35) this becomes 

+ (//W o rr dxd V rl d A S U 

d S W ds \cy) L * d^FxTyV'idj ~ 2K ds ds + L U JJ *• 

If now the curve C is chosen so that 

AlrY-iK±^ +L (±Y-. 0 

ds ds^ l ds) ~ 

then H--dl—\-i.r dx i(‘f\ ,, dx dy 

,h "te/ + L *'%) + M (10.37) 


(10.36) 
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Hd £ d (D +Ld %) +Md!l = 0 - 


(10.38) 


(10.39) 


A curve in the ( x,y) plane such that equation (10.38) is satisfied at each point 
of it is called a characteristic. If K 2 > HL (and only then), the roots of equation 
(10.38) at any point ( x,y) are real and different, so that the characteristics are real; 
it is for this reason that the sign of K 2 -HL is taken as the defining property to 
distinguish the classes of‘elliptic’, ‘parabolic’, and ‘hyperbolic’ equations. Since 
for hyperbolic equations the roots of (10.38) are distinct it follows that through 
each point of the (x, y) plane there pass two characteristics. Thus there are two 
sets of characteristics covering the (x, y) plane, one membei^ of each set passing 
through each point (x, y). These two sets will be called ‘ set 1 ’ and ‘ set 2 . 

If H, K, and L do not depend on /, Bf/Bx, or B/JBy (though they may depend on 
(x,y)), the characteristics are independent of the particular solution, and can be 
evaluated over the whole relevant domain of the (x, y) plane before the evaluation 
of a solution is started. But when one or more of H, K, and L depend on/, J/ x 
or df/dy, the characteristics depend on the solution and the evaluation of 
characteristics has to proceed simultaneously with that of t 10 so ution. 

It fa convenient to write p, for the roots dy/dx of (10.38) the valuof>, 
at any point referring to the characteristic of set 1 through that pomt, and the 

value of ii t to the characteristic of set 2. Then 

y. i+/x 2 = 2K/H, y.y, = L/H. 


On a characteristic of set 1 we have 


dy 


and from (10.39), d {%) + ^ d (dy) ~~ 


% 

on a characteristic of set 2 


d'J 

dy 

dx 


= ht 


“(DMg) = 


(10.40) 


(10.41) 


(10.42) 


(10.43) 


and 

It sometimes happens that the derivatives 0//8x J'j/don^t 

are the quantities required in the solution, and further that f' e cuS0> for 

depend on /, though they may depend on d/J y urn^ c/,^ ^ ^ £//£*, , the 

example, if / is the velocity potential of ’ u . cd Then it is convenient 

components of the velocity, are the quantities reaU> bccomo 

to write u, v for dj/dx, BJ/By respectively, and (10.41), t 

du+y t dv= -(MIH)dx > 

on a characteristic of 6et 1, and 

-<w* <10 - 45) 


on a characteristic of set 2. 
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10.81. Finite differences between characteristics 

One way of adapting these equations for numerical work is, in effect, to use 
members of the two sets of characteristics as defining a finite-difference grid in the 
y ) ptane and to work in terms of finite differences between neighbouring charac¬ 
teristics. In Fig. 27 the two sets of curves represent the two sets of characteristics; 
tho heavy portions represent the portions on which the solution has been carried 

out, and we want to determine the solution on the set of intersections of which A 
is typical. 



0,1 tl,e charac,ori » ,fo A 8 of set 1, a finite-difference approximation to (10.40) is 

i/A-yu = 

and similarly on AC y A -y c = J(p s „ +(i , s K^-x c ). 

Also an All a finite-difference approximation to (10.44) is 

and similarly on AC 


("a-u c )+ j(p,.t+p 10 )(^ - ,. c) = -miim A +(M/H) c ](x A -z c) . 

ro°u*~Z 4 V ’ *' y bCing knOWn *“ B and C - «*»'» * «* Of four equations 

8 f&y these four equations have to be solved as a s,»t nf i, 1 ■ ’ 

Vj.Vj X, i/fthtn-flronnni; . 1 f &,mult aneous equations for 

Af 1 a* Va* nre non-linear and can onlv bo anlvorl K*- „ i 
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11.1. Summation of series 

In practical applications of numerical analysis, as distinct from artificial 
examples constructed for the purpose, it is comparatively seldom that 
the original formulation of a problem is the summation of a series, though 
summation of a series is sometimes a useful method of dealing with a 
problem originally formulated in some other terms. 

For example, the properties of the Airy function Ai(x) which make it 

important in applications are these: 

(i) it is a solution of y" = xy which tends to zero as * tends to 
infinity; this defines it except for a constant multiplying factor; 

(ii) it is | cos(x<-}-$< 3 ) dt. 

It can be evaluated from either of these properties without the use 
of a series expansion. Its power series expansion is a further property 
which happens to be useful in tire evaluation of Ai(r) for small values 
of x, but it is not the primary reason for the importance of this funct ion, 
nor a property which need be used at all in its evaluation 

A series is useful in numerical work only if the sum of the first few 
terms is an adequate approximation to the sum of the series, or to the 
function represented by the series-just what a dew term, and an 
‘adequate approximation’ mean will depend on the context. Suppose 
wo have a numerical problem originally formulated m some other way 
than the summation of a scries, and in trying to e\ana r u , ‘ ' 

summing a series we find that the convergence of the first few ' 

not rapid enough for them to be useful. Then this is a stronghmt that 

evaluation of the series is not the best process foi go i g 
required, and the possibilities of other processes should 

^ButTsometimes we may be concerned with the summation of slowly 
convergent series either in calculations originally formulated in s 

terms, or through the reduction of a more complies ted ' ^ 

a summation. In such eases we need processes lor transforming s.owb 

convergent series into more rapidly conveigen ones. . 

such transformation is one due to Euler for a senes of terms of alt er,,. g 

signs. 
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11.11. Euler’s transformation for a slowly convergent series of 
terms of alternate signs / 

This transformation can be derived by an application of finite-differ¬ 
ence operators, and is one of the few cases in which the use of forward 
differences gives the most convenient form for results. 

Let the magnitudes of the terms be u 0 , u lt .in general u„, so that 

the senes which we wish to sum is 

S = u 0 ~ ttj-f tt 2 — U 3 +... = 2 (—l) n w n . (11.1) 

n 

Let us take the successive differences of the terms u n , regarded as a 
function of n. Then in the notation of §4.6, 

u n = E n u 0 , E = 1-J-A, 

where A is the forward-difference operator with respect to n, defined 
by A u n = u n+l -u n . Then 


.S’ = (1 -E+E*-E*+...)u 0== 




w o = 


"o = Kl + IA)-*^ 


! + 2+A 

= *[« 0 —iA 3 M 0 +...]. (11.2) 

The differences involved here are the forward differences from the first 
f 11 °* ^ ie °f w „. and are all available. 

If the senes (11.1) is slowly convergent, then the successive differences 
of the u„ s usually decrease rapidly and the series (11.2) converges 
much more rapidly than the series (11.1) It will often be best not to 
carry he transformation back to the beginning of the series to be 
evaluated, but to calculate separately the sum of the first N terms 
(A _ 0 or .s, perhaps) and apply the Euler transformation to the 
remainder. A good check on the results can be obtained by carrying out 
this process with two different values of .V, 

Elam P ,c - T o calculate S(x) = V (-l)-/( i+ „ 1) = f or * = I0 

m ~0 

*=:°- t - 0,1)10 “' d - t3 

marked in the sixth clifforoncos. The value oVs? !'° lm ' lin e erTors _ aro becoming 
differences of — l/( x t n) 2 If H r c ^ * 10 sum altcrna te first 

to, two terms of andn^, v ^‘l <that ‘ he 

of the series, we obtain * U LF tIan ^ for mation to the remainder 

ht’S(10) = I7355 + J[69.44 + l ( I0272) + i (2120) + i(544) + A (162) + 

= 17355 f 37595 = 54950. d &(52) + *( 14) + ...] 

The values of differences used are those underlined in the table. 
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m 10 7 /(a; + m) 2 

0 100000 

— 17365 

1 82645 4154 

-13201 -1225 

2 69444 2929 416 

— 10272 —809 —151 

3 69172 2120 265 48 

— 8152 —544 —103 

4 51020 1576 162 51 

— 6576 —382 —62 

6 44444 1194 110 14 

-5382 -272 -38 

6 39062 922 72 19 

— 4460 —200 —19 

7 34602 722 53 0 

— 3738 —147 —19 

8 30864 575 • 34 

-3163 -H3 

9 27701 462 

— 2701 

10 25000 

If we take the first four terms of the series and apply the Euler transformation 
to the remainder, we obtain 
10 7 5(10) = 17355+10272 + 

+ i[51020 + i(6576) + i(1194)+i(272) + *(72)+ *(19) + ...] 

= 27627 + 27323 = 54950. 

This agrees with the value already calculated, and we obtain the 
5(10) = 0-005495 to six decimals. 

If the ratios of successive terms u n+l /u n are nearly constant, a modifi 
form of the Euler transformation can be used effectively. t 

v n = P n u„, 

P being a number chosen so that the variation of v n w ith n is small. T 
S = v 0 -p- 1 v 1 +p- 2 v 2 -P~ 3 v 3 +... 

= [l-(Elp)+(EIP) 2 -(E/p) 3 +~.]v 0 

1 P „_£_/ 14-^1 V, 


1 + {EIP) V ° . 


j? = —— fi + rtxr) 

(/3+l) + A 0 0+ 1 ' 


p f 1 A . 1 a2.i_ \ _A 3 y 0 +- • ( lI - 3 ) 

= ( ^ H)3 

11.12. Use of the Euler-Maclaurin integration formula in the 

summation of series r^^oiiv 

When f(x) is a function such that f f(x) dx can be in ® , fo ’ 

the Euler-Maclaurin formula (0.22) can often be used effect,ve b 

6353 K 
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evaluating sums of the type £ f(m) over a set of integral values of m. 

771 

From formula (6.22) with x 0 = 0 and interval (Sr) = 1, we have 

/ 0 +/ 1 +—+/n 


= J /(*)rf*+£(/ 0 47»)+&/W^^ 

° (11.4) 

and in particular, if/(r) and all its derivatives tend to 0 as x tends to oo, 


~ f (11.5) 

As in the previous section, the result of using this formula can be checked 
by applying it to the series formed by omitting the first few terms from 
the series to be summed. 


00 


Example: To evaluate £ l/(8 + m) 2 . 

m - 0 


For 


CO 

/(*) = l/(a+x) 2 , we have \f(x) dx = 1/a, 


and /'(O) = -2/a 3 , /"(O) = -24/a 5 , 

so evaluation of formula (11.5) for a = 8 gives 


/’( 0) = -720/a 7 , 


CO 


Also 


2 1/(8+ m) 2 = i + 1 —+ — - _L 

m -o 8 ‘ 2 64 ‘ 12 8 3 720 8 5 + "‘ 

= -125+ 00781 25 + ‘00032 55 —-00000 10 + ... 
= -133,137 to six decimals. 

fl/(8 + m) 2 = -L+- 8 » f + f l/(10 + m) 8 

m “ 0 


and evaluat ion of formula (11.5) for a = 10 gives 


I 1/(10 + ?/j) 2 = -L+I JL 4 .I J. _L Ji 

m-0 10~2 100^12 10 J 720 10 s 

= • 1 + -005 + -00016 07 — -00000 03 
= -10516 64 

CO 


SO 


*r 




£ 




^b'fS + w) 2 = -01562 5+ 01234 57 + -10516 64 

= ’13313 7 to six decimals 

verifying the value obtained by evaluating formula (11.5) with a = 8. 

Slowly convergent series of positive terms which cannot be handled 
y 11 s application of the Euler-Maclaurin formula are often difficult to 
deal with numerically. If the terms are given by an algebraical formula, 
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then it may be possible to lind an analytical transformation which con¬ 
verts the series into a more rapidly convergent one, but this procedure 
is not usually available unless each term is of a relatively simple form.f 


11.2. Harmonic analysis 

Harmonic analysis is concerned with the representation of a function 
/(*), over a finite range of z which will be taken as 2 tt, as a series of 

circular functions of x : 

f{x) = £.4 0 -f- -4 i cos a; +.4 2 cos 2a*-}-...-}--#! sinx+H 2 sin 2a;-f .... 

( 11 . 6 ) 


The most important applications are to cases in which/(a) is periodic 
in a with period 2*r, or in which f(x), although not periodic, or not 
defined outside a range x 0 < x < (x 0 +2~), satisfies the com ltions 

/<*>(a 0 +277) =/ ( %r 0 ). ( 1L7) 

If J(x) does not satisfy such conditions, or if it has discontinuities m 
magnitude or in a differential coefficient of low order, then harmonic 
analysis is usually of formal rather than numerical interest, since 
numerical work only a finite number of coefficients in the series (11.4 can 
be evaluated, and the sum of any finite number of terms gi\ es a unc ion 
which satisfies the conditions (11.7) and has no discontinui > 
derivative. Iff itself has a discontinuity or does not satisfy 

/(*©+ 2 ”) =/(*«)» 

then the behaviour of the sum of a finite number of terms, in the 

neighbourhood of the discontinuity (or of an( x o ) r* m7T c 1 
siderably from the behaviour of/(*); as n increases the behaviour off 
remains of the character shown in Fig. 28, the scale of r, but noi the scale 
of the oscillations in/, becoming smaller as n increases. . • 

as the ‘Gibbs j.henomenon’ and illustrates the ncet or ca 
regarding a finite number of terms of the series (11.6) as an adequate 
representation of the function unless it is free from dmeontmurt.es and 

the conditions (11.7) are satisfied. „ rtnn p V mn 

The most usual applications of harmonic analysis are " l , Jv 

with the analysis of results of experiment or observation. 

TCP. Miller, Phil. (") 

t For other methods of treatment of aucli senes, 436* G. G. Macfarlane, 

22 (1030), 704. T. M. Cherry. free. Cm*. f ^ 28 (1949), 273. 

Phil . Mag. (7) 40 (1949), 188. See alao O. Szdsz, Journ. Main, ana y 
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however, it may be required in purely analytical or numerical contexts. 
For example, in the solution of Laplace’s equation in two dimensions, 


d*V d*V 

dx 2 + dy 2 



in the interior of the unit circle, it may be convenient to use the result 
that V can be written 


y = K)+?'(^liCOS0_|-5 1 sin0)+r 2 (;4 2 cos20-{-.g 2 sin20)-f.... 


Harmonic analysis of V as a function of 6 on the unit circle givqs V 0 and 
the coefficients A n and B n directly, and hence the solution V, without 
requiring numerical integration over the whole interior of the unit circle. 



Tin- coefficients in the series (11.0) are given by 

2 7 2tt 

nA " = J f(*)cosnz dx, 7tB h = I /(jr)sin /jo: dx. (11.8) 

u o 

It f(x) satisfies the conditions (11.7) it follows that, in each of these 
integrals, each derivative of the integrand has the same value at the 
upper limit as at the lower limit. Hence in the Euler-Mnclaurin formula 
(().2.») for each of the integrals the correcting terms from the two ends 
of the range cancel identically, and if the integrals can be evaluated 
rom a set ol values of f(x) at equal intervals on a-, the appropriate 
integration formula is the trapezium rule without corrections. Hence 
if the range 2n in r is divided into K equal intervals, we have 

t' • 


\ K .-! 


K -l 

i /(•<■/.)eos n. tv, 

A ? 


where x,. — 277 k K. 


K- 1 

• K = 2 /(a- A .)sin n.r k , (11.9) 

A-0 
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These expressions for the coefficients are not significant for n > 2 K. 
This can be seen as follows. Let n 0 be a value of n less than 2 K, and 
m any positive integer. Then at the points x k = 2-rrk/K we have 

cos {mK±n 0 )x k = cos{2nmk±n 0 x k ) = cos n 0 x k , 

sin{mK±n 0 )x k = sin(27rmk±n 0 x k ) = ±sin n 0 x k . 

Hence at these points the contributions from the terms with 
» = mK±n 0 have exactly the same variation with k as contributions 
from the term with n = n„; and no analysis using only the values of/ 
at these points can distinguish the contributions from the values 
. = mK±n 0 with different values of «. The first of the sums (1.9) 
gives the same values for each A n (n = mK±n„y, but the values of the 
A n ’ S are independent, so these values given by (11.9) cannot all be s.gmfi- 

Ca The point is that for values of n greater than 1 A the values of** are 
not closely enough spaced, relative to the period of «»**,.forthe 
formulae (11.9) for the integrals to be valid. For » = U the values of 
the integrand /(*)cos nx at successive values of x k are 

+/(*„). —/(*!»• +/(**)■ - 
and for most functions / these values are too irregular to give any 
confidence that they represent the behaviour_of t '^“^verge 

enoughtoiustifyanynumericalworkonitatall.Their i 

and the situation is similar to that considered in § 6.54 where also w 
were concerned with an integral for which the correction to the teapmum 
rule vanished at both ends of the range of integration, but an ncorrect 
value was obtained if too great an interval of mtegration wa akem To 
define an oscillating function adequately it ,s advisabte to have a jeas 
six points per period, and this suggests that the series (11.9) should no 
be regarded as adequate approximations to the integrals (11.8) for 

n Another aspect of these results can be illustrated by ^ering^an 

alternative way in which the coefficients A,„ „ ">ig 

numerically, not as values of the integrals (118, but by 
(11.6) to f(x) at the discrete set of values x k o x. 

have 


K y f{x k )coBnx k = lK[A n + A K -„ + A Kin -\- A 2 K-n-\--] 
k -0 

K £}(x k )a\nnx k = \K\_B,-B k J 

A;—0 


( 11 . 10 ) 
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It is clear that with the restriction to K values of x, it is not possible to 
determine more than K relations between the coefficients, namely the 
first of relations ( 11 . 9 ) for n = 0 to \K and the second for n = 1 to 
1); the terms B n sin nx k for n = {m-\-%)K make no contribution 
to the sum ( 11 . 6 ) at any of the points x = x k . The smoothest function 
with the assigned values of f(x k ) will be that for which A n = B n = 0 
for n > and then A n , B n for n < \K are given by ( 11 . 9 ). 

If f( x ) is a continuous function, then a good test of the significance 
of the values of A n , B n calculated from (11.9) is given by making two 
analyses with values of K which are relatively prime or have only 
a small common factor, such as K = 30 and 32, or 48 and 50; this process 
also provides a good overall check on the results. 


11.3. Recurrence relations for a sequence of functions 

The Bessel functions of integral order J„(x) form an example of a set 
of functions of one variable (x) and one parameter (n) which have a 
number of properties in common, such as the form of the differential 
equation satisfied by them and their asymptotic behaviour. They are 
connected by relations between the functions of different orders n, 

such as T / \ r 

4,+i(*)-4,-i(s) = —2J n (x), (11.11) 

J n-\ M+ J n +1 (*) = (2nfx)J„ (.r). (11.12) 

Such relations are called recurrence relations. Other examples of such 
sets of function are the Legendre functions P„(x), the confluent hyper- 
geometric functions If A ,„,(.r) of Whittaker,f and the Weber functions 

A,(*)t 

It is often convenient to use such recurrence relations to evaluate 
functions, for some value of the parameter for which there may be no 
tables available, from tabulated values of the functions for other 
parameter values. Such a process must be used with care or it may lead 
t o quite spurious results. This can be seen by considering, as an example, 
the evaluation of Jjx) for a given value of x and for large values of n 
from ,/ 0 (.r) and J t (.r) by repeated use of the relation (11.12). 

For •/„(.<•) we require that solution of ( 11 . 11 ) which tends to zero as 
n tends to infinity. But if we evaluate J n (x), J n +1 (.r), J„ +2 (r),... in succes¬ 
sion by using (11.12) in the form 

J nn ( x ) — (-n/ar )/„(!■)-•/,,_ 1 (a'), (11.13) 

the rounding errors introduce a small multiple of the second solution 
t fbil; r^ k0r " nd C " N - "'“ ,SOn ’ •■»«<%•» (C.U.P. 1927), cl.. 16. 
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Y (*) of this recurrence relation. For n < * this remains small, but 
for n > x it behaves roughly as an increasing exponential, and increases 

without limit as n tends to infinity. . f 

Thus this way of using the recurrence relation is not satmfactory fo 

calculating Bessel functions JJx) for . > «. though 

for n<x. It would, however, be satisfactory for ca culat.ng YJx) 

since in this case the unwanted solution, of which a small multiple may 
be introduced by rounding errors, is one which decreases indefinitely, 

relative to the wanted solution Y n (x), as n increases. 

On the other hand, the range over which («/*) > 13 , ti 

over which relaxation methods can be used effectively fo the * 
of (11.12), provided the solution has to satisfy two-pointboundary 

conditions in n. This is the case for the Bessel function ( ), P 

to » = x these can be built up satisfactorily by successive o 
(11.13). This gives JJn) as one terminal condition in n, and the other 

is given by JJx) -* 0 as n -> oo be obtain ed by 

This example shows how quite different rtsu 

different ways of using the same simple formula; one way °f 
recurrence relation (11.12) may lead to quite 6 P u " OUS ; e ;;‘ ^‘tay of 
no mistakes have been made in the^—,- any 
using the same formula can be used g 

assigned degree. 

o.. 

» .. property of wh.rh “ ” of hi 8 lr-order 

tion. For a continuous function it I imulies smallness of 

derivatives, and for a table of function u ..inritv of the differ- 

the higher orders of differences; this *"*P''“ “ S ° h " (n+ 10)th differences 
ences, since if the nth differences are irr ^ > 

will not be small. . . nroce6S of replacing 

By‘smoothing’a set of function va ues is i ^ ^ b ut are 

them by another set which differ on y »‘ f function values has 
‘smoother’ in this sense. If each mem er o <■ . g subject to a 

been obtained by an independent calculation, an < jncicas j d so me- 

rounding error, then the accuracy of ^.^ vinent shou ,d not be 
what by a smoothing process. Bu • be re , ied on because 

relied on, and cannot be estimated. b ° num bcr of consecutive 

it is always possible that the rounding errois ina gnitude, and 

function values may be of the same sign and similar in 
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then smoothing will not improve them. Further, as we have already 
seen in Chapter IV, an incorrect set of function values may have smooth 
differences, and a set of correctly rounded-off function values may be 
less smooth than a set obtained by rounding off incorrectly. Also 
without knowing in some other way a more accurate set of function 
values, there is no criterion by which the improvement of the function 
values can be assessed; and if these more accurate function values were 
known, there would be no point in carrying out the smoothing process. 
If a set of function values is too much affected by rounding errors, the 
only reliable way of getting more accurate values is to carry out the 
calculation of the function values to greater numerical accuracy. 

The main purpose in carrying out a process of smoothing must there¬ 
fore be to achieve smoothness, not accuracy. The contexts in numerical 
analysis in which smoothness is a prime requirement are not many, so 
that such a process is not often required. But occasionally it is difficult 
to make satisfactory progress without one. 

Consider, for example, the evaluation of a set of solutions of a differen¬ 
tial equation involving a function f{y) determined by experiment or 
by statistical sampling, the different solutions being distinguished by 
different initial conditions or different values of one or more parameters. 
For consistency between the various solutions, and also in order to use 
the differences of intermediate quantities for checking the integrations, 
it may be advisable to use in the numerical work a table of f(y) which is 
smooth to a substantially greater degree of numerical accuracy than the 
accuracy of the experiments from which f{y) is determined. This can 
sometimes be achieved by fitting an analytical formula to the experi¬ 
mentally determined values; when this has been done, the formula can 
be evaluated to any required numerical accuracy. But this process is 
inconvenient unless a relatively simple formula can be found to fit the 
experimental values within the experimental or sampling error, and it 
is also unnecessary. A more purely numerical smoothing process is 
often more useful and more effective. 

Another example is provided by the process of § 10.72 for the integra¬ 
tion of a parabolic partial differential equation. For one time interval 
of this process, as applied to the equation 

dt- dx 2 ’ 

the equation 
§2 

^[/0M+SO+/(*,0] = (2/Si)[M<+S()+/(a-,0]-(4/Si)M0 
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is solved with two-point boundary conditions in *. In the processes of 
numerical integration and interpolation for the solution satisfying the 

tV lnz?+S+n* T W up n to ±2 in thela" slgnitant figure kT P T If the 
Z X. « Richardson's process of spols.lon i. ««l •- 

=£ s rrz;”d ri”r rz 

Sissssssii 

rln “^dvi»bt1««” P nally to smooth /(*,«) as a function of * 
during the progress of the calculation. 

11.41. Automatic methods of smoothing . 

a • i ™ nlP of one class of methods of smoothing is the following. 

A simple examp - fiye succe ssive values 

Replace each function value £ by the mean j, 

of / centred on £, that is, take 

Th „,o r i r 

by the set of function values 

f 00001000 0, (11 14) 

for which this process gives 

/ 0 0 J 1 1 1 1 0 0. (11.15) 

The maximum value of |8 2 /l is i whereas that of |S 2 /| is 2: on the basis 
If this criterion, the set of values (11.15) is ten times as smooth as the 

SC Thil process of smoothing by groups of ( 2 n +1) can be repeated For 
example, two successive processes of smoothing by groups of tlm , 
starting from the set of values (11.14) gives 

/ o o i l 8 I i 0 o- (111G) 

For this set of values, the maximum |5 2 /| is l 
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These are two examples of a general method which consists of replacing 
each fj by a linear combination 

k= —n 


of function values centred on The smoothest set of function values 
is simply /• = constant, and in order that these should not be altered 
by the smoothing process, the coefficients must satisfy 


2 i; 

— n 

and normally the coefficients will be symmetrical about k = 0. Different 
processes are given by different choices of the coefficients a k in (11.17). 

These methods are often unsatisfactory in practice for three reasons. 
First, once the particular smoothing formula to use has been decided, 
it is automatic in character in that the results are then determinate, and 
gives no opportunity for the exercise of judgement by the individual who 
is carrying out the calculation. This might at first sight seem an advan¬ 
tage, since the results will then be independent of the individual. But 
this apparent definiteness of the results is spurious since there is a good 
deal of latitude in the choice of what smoothing formula to adopt. And 
the smoothing process is in practice one in which it seems desirable to 
give the individual who is carrying it out some discretion on matters 
such as the degree of smoothing at which to aim and the degree to which 
changes fj—fj from the original function values are acceptable. Secondty, 
with methods depending on the use of formulae of the type (11.17), the 
smoothed values J } cover a smaller range of j than the original values; 
in a method due to Spencer,f recommended by Whittaker and Robinson, X 
ten values at each end of the range are lost, so that from 30 values of 
fj only 10 smoothed values in the middle of the range ofj are obtained. 
Such a loss of range is often unacceptable. Thirdly, a special procedure 
is needed if it happens that some value of f(x) is known exactly, such as a 
value f(x) — 0 at x = 0, and is not to be modified by the smoothing 
process. 

The dangers of a blind use of an automatic smoothing process are 
illustrated in Fig. 20. Here the full curve is representative of the be¬ 
haviour of the function f(v) — Rjv 2 , where R is the resistance of the air 
to a body moving through it at a speed of v ft ./sec. If f(v) is tabulated at 
intervals of 50 ft. sec. the behaviour of the second and higher differences 
of /(f) is rather violent, and can be mollified hy the application of a 


t J- Sponcer, ./. Inst. Actuaries, 38 (1004), 334. 

I Calculus of Observations (Blarkio, 1940), p. 290. 
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smoothing process. Spencer s process, applied to these data, gi\es 
results represented by the squares and broken cur\e in Fig. 29. They 
are certainly smoother (the greatest value of | S 2 /1 has been reduced 
from 232 to 48 in terms of the third decimal place as unit). But it does 



Fig. 29. 

not follow that the smoothed values are a better representation of the 
actual behaviour of/(») than the unsmoothed values; they are a mos 
certainly worse, and in particular the minimum about i ’ 8 ^. 

is almost certainly spurious. But if one insists on using an automatic 
formula one has no control over the results it is going to £ 1VC > 1 
believes the use of the formula to be significant at all, all that one can 

do is to accept the results of using it. 

11.42. Smoothing by use of an auxiliary function 

A less formal but more practical method is due to A. I. PoocLson.t 
It is based on the use of graphs. 

Let }(x) be the function which it is desired to smooth. Lnless J(x) 
is of only two or three figures, and sometimes even if it is of three figure 
it will not generally be possible to smooth it directly by plotting and 
drawing a smooth curve ‘through' the plotted points. But if yM is 

t Thi, moll, oil ~ d.vi«d in 3( 1W. I'h" 

War, but only published rocontly, in Quart. J. Mech. ana yipf 
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smooth function approximately equal to f{x), it may be possible to plot 
the difference f(x)—g(x) on a scale open enough to smooth it, to the 
degree of numerical accuracy required, by such a graphical process. 
Then/(a;) 1 , the smoothed function by which f(x) is replaced, is constructed 

88 /(*) = < 7 (#)-fsmoothed{/{:r)— g{x)}. 

The auxiliary function g{x) can be formed in several ways. It may, 
for example, be taken to be given by an analytical formula, such as 
ax 2 , ax/(x 2 -\-b 2 ), e™, 6e -ar *, if there is any theoretical reason or empirical 
indication that f(x) is approximately of such a form. Another process 
is to build up g{x) from a smooth set of differences. This is of more 
practical use in many cases, as it can be used equally well whether or not 
a good approximation to f(x) can be obtained by a simple analytical 
formula, and it does not involve any selection and adjustment of para¬ 
meters in an analytical formula so as to get a good overall fit to f(x). 

For simplicity suppose f(x) to be given at equal intervals of x. And as 
an example of the general process, suppose this function, and the 
interval of tabulation, to be such that the range of the values of 8 2 / is 
not more than 200, so that they can be plotted on such a scale (1 mm. 
or in. to a unit) that they can be read off to a unit. 

The process is then as follows. Plot S 2 /(.r) and draw ‘through’ the 
plotted points as smooth a curve as possible without smoothing away 
significant features of the behaviour of the second differences. Tin's is 
<>n< place at which discretion is required in judging what features of the 
behaviour of t lie second differences are significant. If the uncertainty 
of each value of/(.r) is known, the range of uncertainty of each second 
difference can be found and indicated on the plot, amfthis may help in 
distinguishing significant from non-significant features of the variation 
of t he second differences. It is better to over-smooth at this stage rather 
than the reverse; significant variations which are smoothed out at this 
stage are replaced at a later stage. 

Let h(x) be these smoothed values of 8 2 /(.r). It is not advisable to 
double-sum them directly to give the auxiliary function g(x). A small 
systemat ic difference between two different ways of drawing the curve 
from which fi(.v) is read off may build up, on double-summing, to a 
substantial amount, so that though one curve might give a g{x) which 
differed little trom/(.r), the other might give a g(x) departing from it to 
such an extent that it would be difficult to plot/(.r)-< 7 (*) on an adequate 
scale; and there can be no certainty that the curve from which A(.r) is 
read off is not of the latter kind. It is therefore best first to form the single 
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sum ah{x ) of the values of /*(*), and to modify this if necessary so as to 
get a general agreement with the first differences of f{x), before forming 
an auxiliary function. The differences 

8/ (x) — ah{x) 

between the first differences of/(x) and the first sum of h{x) are therefore 
plotted and smoothed graphically. The smoothed values of 

8 f{x )— crh(x) 

are then added to the values of ah{x) to give the first differences of the 
auxiliary function g(x) which is then built up from these differences. 
Thus g{x) is given by 

g = a[aA + smoothed(8/— ah)]. 

Finally, {f—g) is plotted and smoothed, and the smoothed function 
f{x) is given by 

J(x) = g{x) +smoothed{/(x-) — J 7 (x)}. 


In this final stage discretion can again be exercised regarding the 
extent to which values of / may be modified by the smoothing process, 
and the significance of various features of the behaviour of /. The 
difference between the original and smoothed values of / at any value 

of f 13 f{x)-J{x) = {f(x)-g(x)}-smoothed{f{x)-g(x)} t 


and the right-hand side here is the departure of the smooth curve from 
the plotted point {f—g) at each value of c. If the range of uncertainty 
of each value of/, or the maximum change in each value which would be 
acceptable, is known, this can be indicated on the plot, and the smooth 
curve drawn so that its departures from the plotted points do not 
exceed this range at any point. In particular, if at any point the value 
of f{x) is known exactly, the curve of {f—g) must be drawn to pass 
through the plotted point at that value of .r. In Doodson’s method of 
smoothing, particular features such as this can be taken into account 
quite easily and without departure from the regular procedure. 

The differences of the final values of /(*) provide a check of the cal¬ 
culation and an indication of the degree of smoothness which has been 
obtained. An indication of the extent to which the final results depend 
on the details of the process of smoothing is given by currying out tlu 
process twice using the same set of values of f(x) but difTeient smooth 
curves from which to read off the smoothed second differences of h{x). 



254 


MISCELLANEOUS PROCESSES 


The process can be adapted to start from other orders of differences 
than the second. However, if f(x) is at all seriously irregular, the higher 
differences of f(x) probably vary so wildly that it is difficult to see any 
general trend in their values. 

Examples of this process, and its extension to functions of two 
variables, can be found in Doodson’s paper. 



XII 

ORGANIZATION OF CALCULATIONS FOR AN 

AUTOMATIC MACHINE 

12.1. Automatic digital calculating machines 

When we write a number in the ordinary way, such as 1925, the symbols 
such as 1, 9, 2, 5 in this example stand for what we call the digits of a 
number, and a piece of equipment which operates d.rec y with and 
records, the discrete digits of each number is often called a digital 

calculating machine. Since about 1938 there has been a devel ° P n 

ment of such machines with two important features. First, they can 
carry out long and intricate numerical calculations quite automatically 
once they have been provided with a specification, in a ^table form, 
of the calculation to be carried out. And, secondly, t ey 
versatile, so that the same machine can be used for many quite different 
kinds of calculation; for example, for calculating values of a function 
from its power-series expansion, for solving large systems of linear 
simultaneous equations, for finding the characteristic values of matrices, 
and for the step-by-step integration of ordinary differential equations^ 
To express these two features, such machines are sometimes called 

general-purpose, automatic, digital calculating machines. 

The process of organising calculations for such machines ,s a branch 
of numerical analysis which has only come into being with the machines, 
and this chapter is included here to give an introduction to the subject 
It is concerned with the planning of calculations for such machines 
rather than with the machines themselves; it is only concerned with the 
machines in so far as their characteristics affect the process of organising 

calculations for them. . nf i 

Wc have already considered in §2.7 the general organization of a 

hand computation done with the assistance of a desk machine. A 

automatic machine must be capable of carrying out the same processes 

and can be thought of as having a similar organisation, as shown 

diagrammatically in Fig. 30. It must have an arithmetical unit m wl .c 

arithmetical operations can be carried out, to take the place of the des 

line » ! hL. calculation; a both to, nun*.,. and or 

control system to take the place of the human computer t 

sequence of operations in a hand calculation. The machine also needs 
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input and output equipment for receiving numerical data and operating 
instructions from the outside world and for delivering its results. 

Whatever the physical form of the store, it must provide a number of 
identifiable storage locations, and it is convenient to think of these as 
numbered. The number which is the label of any storage location is 
often called its ‘address’, or the ‘address’ of its content. C(n) will be 
used for ‘the content of storage location n\ It is sometimes convenient 
to represent the address of the number which is the value of a quantity 
x by L(x) or A(x), or by C _1 (a:) if it is desired to emphasize that the 
relation n — L{x) is the inverse of the relation x = C(n). 



Fio. 30 


The specification of an operation which the machine is required to 
carry out is called an ‘instruction’ or ‘order’ (sometimes a ‘command’), 
and the ordered set of such instructions needed to cany out a calculation 
is called the ‘program’ for that calculation. An important feature of 
most recent and projected machines is that instructions are coded in 


such a way that they have the same form, within the machine, as 

numbers, the difference between numbers and instructions being in the 

way they are used. The content of a storage location is then usually 

called a ‘word’, whether it represents a number or an instruction; ‘words’ 

representing numbers are normally used by being transferred between 

the store and the arithmetical unit, and ‘words’ representing instructions 

are normally used by being transferred to the control system. But since 

there is no distinction, within the machine, between numbers and 

instructions, it is possible to use the arithmetical unit to build up, 

transfer, and modify the operating instructions themselves as the 
calculation proceeds. 

The effect of this on the organization of a calculation is so profound 
that there may be little relation between the processes of organizing a 
calculation for machines which do and for those which do not provide 
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this possibility. However, its importance is now well realized, and it is 
provided on several of the machines already (1951) in operation and is 
likely to be provided on all future machines. It will be assumed in 

the rest of this chapter. 

There are two main forms for instructions; these can be illustrated 
by an example. Suppose we want the machine to form the sum of the 
contents of storage locations n x and ?i 2 and to put the result into location 
n . This could be done by a single instruction which could be written 

o 

symbolically C(n x )+C{n 2 ) to w 3 . t 12 - 1 ) 

An alternative is as follows. Suppose that the arithmetical unit is of a 
kind which includes as one of its components a register, usually called 
an ‘accumulator’, corresponding to the 'product register’ or ‘accumu¬ 
lator’ of a desk machine, which accumulates the sum of numbers added 
into it until it is cleared ; the content of the accumulator will be written 
C(Acc). Then the required operation can be done by the three separa e 


instructions: 

C(n x ) to Ace, C(n 2 ) to Acc, C{Acc) to w 3 . (12.-) 

Each of the instructions of the form (12.1) specifies three addresses in 
the store, whereas each instruction of the form (12.2) specifies a s.ngle 
such address. These forms of instruction are consequently known as 
the ‘three-address’ and ‘one-address’ forms respectively. 

An instruction specifies an operation to be carried out. But it is also 
necessary to specify the sequence in which such operations are to be 
carried out; that is to say, after carrying out one operation, the machine 
must be enabled to select the instruction for the next operation. In a 
machine in which instructions are contained in the same store as numbers 
this can be done in two ways. One is to include in each instruction the 
address from which the next instruction is to be taken; with a three- 
address specification of the operation to be earned out, this gives, 
altogether a ‘four-address’ form of instruction. Another way is noi m< y 
“ ole instructions at addresses numbered serially in ^ sameonler 
as the time-sequence in which they are to be carried out. 

-referred to as ‘serial storage’ of instructions. In this ease the address 
of the instruction currently being carried out is recorded in a reg'S 
whose content is normally increased by unity on the completion of tins 
instruction, and the content of this register is used t^ontrolthescdec.o 
of the next instruction. Then an explicit specification ofthe mh< sso 
the next instruction is needed only when it ,s required to depart 
the serial order in which the instructions are located in the store. 


MS3 
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The control system of a machine will depend on the standard form 
adopted for instructions and the means adopted for selecting the next 
instruction, and once the machine is built, the instructions used must 
conform to the type for which the control system has been designed. 
These features, rather than the physical form of the store or of the arith¬ 
metical unit, are the essential features of a machine from the point of 
view of the user. 

12.2. Preparation of calculations for an automatic digital cal¬ 
culating machine 

The process of preparing a calculation for an automatic digital machine 
can be broken down into two parts, often called ‘programming’ and 
‘coding’. 

By the ‘program’ for a calculation is meant the schedule of operating 
instructions which has to be provided to the machine in order that it 
shall carry out the calculation. ‘Programming’ is the process of planning 
the sequence of operating instructions required, and ‘coding’ is the 
process of t ranslating these instructions into the particular form in which 
they are supplied to the machine. In simple calculations these are 
hardly two distinct processes, but in more elaborate calculations it is 
convenient to treat them as separate. 

A process of programming is required in a hand calculation; before 
we can start doing any calculation we must decide just how we are 
going to do it. For work with an automatic machine, programming 
may involve breaking down the calculation to a sequence of the elemen¬ 
tary operations, such as addition, multiplication, and selection of the 
next instruction, which the machine can carry out. But the machine 
and the process of providing it with instructions may be such that groups 
of operations for standard processes, such as evaluation of cos* given 

b 

'he value of x, or of J f(x)dx given the values of f{x) at a set of values 

Cl 

of a-, can be programmed and coded once for all. If this can be 
done, each such process can be regarded as a unit in programming a 
calculation, and not analysed into elementary operations. The pro¬ 
gram for such a standard process will be called a ‘sub-routine’. The 
possibility of using such sub routines freely greatly lightens the work 
of preparing a calculation for a machine, and the machine and the form 
of its instructions should he planned to provide this facility. The use of 
the same form in the machine for instructions and for numbers, and 
the freedom which this gives to modify instructions by arithmetical and 
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other operations on them, are important features in making it easy to 
provide and exploit the possibilities of using such sub-routines. To a 
potential user of an automatic machine, means of organizing calcula¬ 
tions for it are as important as the provision of the machine itself, and 
the provision of a library of sub-routines for standard processes is an 

important step in this direction. 

Although the various kinds of machines differ considerably from one 
another in their internal organization and operation, the general process 
of programming a calculation will be much the same for any of them, 
for it depends primarily on the structure of the sequence of operating 
instructions required to carry out the calculation. Some characteristic 
features on an individual machine may, however, affect the details of the 


programming. Such features are 

(i) The standard form of operating instructions adopted; whether 
this is, for example, a one-address or four-address form. 

(ii) The facilities provided by the standard instructions; for example, 
whether division can be carried out directly or has to be done by 
means of an iterative process which has to be programmed. 

(iii) The criteria which it is possible to use for discrimination bet ween 
possible alternative courses of procedure; for example, whether 
it is only possible to discriminate on the sign of a number or also 
on criteria such as the likeness or unlikeness of signs ol two 

numbers. 


The process of coding does, however, depend on the part icular machine. 
It docs not depend primarily on the physical form of the store or on 
the wav in which numbers are represented in the machine, but on two 
features, namely the standard form of instructions and the means of 
selecting the next instruction, as explained at the end of § 12.1. 


12.3. Flow diagrams 

It is sometimes convenient to represent the general structure of the 
sequence of processes required in a calculation by a How diagram. his 
can take the form of a block diagram, in which the blocks represent 
operations or groups of operations, and are joined by directed lines 

representing the sequence of these operations. (fpnt - ... 

Suppose, for example, we want to carry out the numerical nitration 

of an ordinary differential equation in the form 

if + 'lhj'+y{\+y°-) = Ccobx 

through four periods of cosx starting from * = 0, using intervals 
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hx = tt/ 15 = 12°, and printing the results at the end of every period in 
cos a:, that is to say, after every thirtieth interval; and we want to do 
this for twenty sets of initial conditions. The general organization of 
this calculation can be represented diagrammatically as in Fig. .31. 


Snitch A 



Fig. 31. 


Processes 

I. Selection and printing 
of initial conditions. 


II. Integration procedure 
for first interval (if 
different from III). 

III. Normal integration 
procedure for one 
interval. 


IV. Printing. 


Here the blocks containing roman numerals represent different 

processes required in the calculation, the processes being indicated in 

i lie table beside the diagram. The blocks divided into sections represent 

counting and switching processes, each of which has one input and a 

number of outputs. They do not, or anyway need not, represent actual 

switches in the sense of hardware, but they may be groups of machine 

operations—operations of addition for counting and of selection of the 

next instruction for switching—which altogether produce the same 

result as an actual switch would do. Each of the sections of one of these 

blocks represents a switch position, and the number n in it indicates 

that the switch moves to its next position to the right after n inputs to 

the switch in the position labelled ». ‘Re-set’ at the right-hand end 

means that the last input to the switch in its last position re-sets it to 
its first position. 

l‘ur example, each of the first twenty-nine inputs to the switch 
labelled switch <" results in the integration being carried out through 
one interval of integration without printing; the thirtieth input to this 
switch results in the integration being carried through another interval, 
followed by a printing, and re-sets the switch; this group of operations 
controls the printing of the results once every period of cos x. After this 
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printing, the next input to switch C is through switch B, which controls 
the number of periods of cosz through which the calculation is taken. 
In some processes of numerical integration a special procedure is 
necessary for the first interval in order to start the integration, and then 
the procedure for the first period of cosz is different from that for 
subsequent periods, the special procedure for the first interval being 
followed by the standard integration procedure for twenty-eight inter¬ 
vals, without printing, and for a further interval followed by printing; 
switch B also controls the sequence of processes for this first period. A 
the end of an integration control is transferred to switch A for counting 
the number of integrations and for determining the initial conditions 
for the next integration, if there is to be one, and for stopping the 


machine if the work is complete. . 

A flow diagram of this kind may be helpful in showing the general 

structure of a calculat ion and how it can be built up fro,n simpler processes 

for which sub-routines may already be available. For this example 

standard sub-routines would probably be available for processes II and 

III apart from the evaluation of y from y and y according o the 

differential equation, for process IV and for the counting and switching 

processes, leaving only quite a small amount of detailed programming 

and coding to be done for this particular calculation. 

It is also possible to draw more detailed flow diagrams showing he 
sequence of operations within a single process such as those rep, esented 
by single blocks in Fig. 31; but it is simpler to program and code 

simple processes directly. 


In discussing coding it is easiest and clearest to write in terms of 
one particular niachine; the different kinds of instructions and Relent 
ways of selecting the next instruction make it difficult to generalise or 
to illustrate the different possibilities in a short aeeount Unders and g 
of the methods of coding for one machine is a great help ^ 

that of other machines, so that this account, though written in terms ot 

most familiar, and the following therefore refers mainly 
at the Mathematical Laboratory at the University of Cam>, Hg-t 
The main features of it which we need for a discussion ot coding 

(i) It uses a one-address form for instructions; 

t M. V. Wilkes, Nature, 164 (1949), 557; Journ. Sci. In*ts. 26 (1949), 9*9. 
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(ii) It uses serial storage of instructions; that is, instructions are 
normally stored at addresses numbered in the sequence in which 
the instructions themselves are to be carried out; 

(iii) Numbers are stored in binary form ; negative numbers are stored 
as complements, the sign of a number being indicated by a ‘sign 
digit’ (0 for a positive number, 1 for a negative number) in the 
most significant digital position; 

(iv) The system of storage is such that the content of a storage location 
remains unchanged until another ‘word’ is planted there; 

(v) Input is by five-hole teleprinter tape, numbers punched in decimal 
form on the tape being translated into their binary form in the 
machine during the process of reading the tape; 

(vi) The machine does not proceed by reading an instruction from the 
tape, carrying it out, then reading the next, and so on; normally 
the whole set of entries on the tape are first transferred to the 
store, and then the machine starts doing the calculation. 


An instruction is written in the form of a ‘function letter’ specifying 
the operation to be carried out (.1 for add, S for subtract, T for transfer, 
) , cases, la an address to which that instruction 

refers. It is punched in the form of a row of holes representing the 
function letter, followed by rows of holes representing the decimal 
digits of the address, followed by a code letter whose purpose will be 
explained later ($ 12.5 (</)). The following are the written forms of the 
instructions which are involved in the examples here considered:! 

add into the accumulator, 
add ( '{n) into the accumulator. 

transfer ('(Arc) to storage location it and clear the accumulator, 
transfer t’(drc) to storage location n and retain it in the 
accumulator. 

examine sign digit of C{Arc)\ if this is 0 take C\n) as the next 
instruction: if sign digit of C(-4rc) = 1 proceed serially (that 
’ s * it this instruction has been taken from address in, take 


A >/ 
X a 
T n 
V n 

K n 


t J'or tho or.tnploto sot of demontary operations ami the form of the instructions for 

„ 'V' 17 . V‘ ' ,lkos • Wm - Sri - Ins ’*- 26 <■»■*»). -’17; M. V. Wilkes nn<l W. Ronwick, 
\!.\ l ", •4(1 '.*;>«*), 1.1 ; M. \ . \\ ilkos Applied Scientific /iescarch, H, 1 (1950),420; D. J. 

" heeler, J'roe. Hoy. So:. A. 202 (1950), 573. 

1 ho examples here considered hu\o been chosen to use only some of the more important 
instructions so that the reader will not have to mcm-ai/.e the whole instruction code in 
order to follow the examples. For a detailed account of programming for tho EDSAC, 
7? . • "'l 1 ;'’*- J) - J - " heeler, and S. Gill. Preparation of Programs for an 

Electronic Digital ( alculatmg Machine (Addison-Wesley Press, Cambridge, Mass., 1951). 
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C(m+1) as the next instruction); if the word standing in the 
accumulator represents a number, this gives a discrimination 
based on the sign of this number. 

G n examine sign digit of C{Acc)\ if this is 1, take C(n ) as the next 
instruction; if this is 0, proceed serially. 

Z stop. 

The function letter of a written instruction is represented in the 
machine by a group of digits in the five most significant digital positions 
(including that of the sign digit) of the word which represents the instruc¬ 
tion and the address n referred to in the instruction, which is written 
and punched in decimal form, is translated into its binary form in the 
machine as part of the process of reading the tape and converting entries 
on the tape into contents of storage locations. 


12.5. Examples of programming and coding 

(a) To place |C(6)| in location 4. 

As a simple example of the use of instructions of this kind, consider 
a sequence of instructions for placing | C(6) | in location 4, 0(8) being 
in complementary form; this process is to be considered as one component 
process in a larger calculation. The required result can be obtained in 
several ways. The following is one possibility, it is assumed that the 
accumulator is initially clear and that the first of these instructions is 

located at address in: 


Notes 


Address 

Content 

Resulting 

C(Acc) 

in 

A 

0 

C( 6) 

m -\-1 

in+2 

E (m + 4) 
T 0 

0 

w + 3 

S 

0 

— C(0) 

m-\- 4 

T 

4 

0 


tost sign digit of C(Ace) 


Cl 6) is first put into the store, then its sign digit is tested. The subsequent 
procedure depends on whether 0(0) is positive (including «',o) 

Consider first what happens if it is negative; then we "ant -( 00 
to be placed in location 4. In this case the next instruction aft 1 c 
test is taken from address (m + 2), and this and the two subseque 
instructions have the effect of placing -C(6) m location as 

Tfcjo) is fou nd to be positive, then wc want it to be placed in location 
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4; it is in the accumulator at the stage at which the test of sign is made, 
we therefore want the next instruction to be T 4. We have already just 
such an instruction in location (m+4) to deal with the case in which 
C(6) is negative. Thus when C(6) is positive we want the machine to 
take the instruction in location (m+4) immediately after the test of 
sign. Hence the address specified in the ^-instruction in location 
(m+1) must be (m+4). 

The address specified in the instructions in locations (m-f- 2) and (ra-f-3) 
may be any address not at this stage containing a number or instruction 
required later in the calculation. Address 0 is often used, as in this 
example, as temporary storage for numbers which are wanted almost 
immediately in the course of the calculation; it is also a convenient 
‘rubbish-bin’ to which contents of the accumulator which are no longer 
required can be sent in order to clear it. 

The instruction in location (m-f-3) could equally well be 5 6 but then 
two instructions instead of one would have to be changed if we wanted 
to use a similar group of instructions to put into location 4 the modulus 
of the content of some storage location other than 6. 

(b) To add the contents of storage locations SO to 150 inclusive atid place 
the result in storage location 4; or symbolically 


o 


2 C(S0+j) to 4. 

> = 0 

This is to be thought of as one process in a larger calculation in the course 
of which the contents of locations SO to 150 inclusive are calculated, the 
sum being required in location 4 for use in a sub routine which (for 
example) is drawn up in such a way as to find sin{C(4)}. 

The simplest and most straightforward way of programming this 
would be by the sequence of instructions: 


A 80, A 81, A 82,..., A 140, A 150, T 4. 

This supposes the accumulator to be initially clear as a result of previous 
instructions. If it is not. it is first cleared by the instruction T 0. 

This sequence of instructions, however, is very extravagant of storage 
space, and as far as storage space for instructions is concerned the 
calculation can be done much more economically as follows. We take 
the group of instructions 

A 4, .4 n, T 4 

which result in C( 4) being increased by C{n), and use this group of three 
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instructions over and over again, with n having successively the values 

80 81. 150. This means that we have to increase the value of n, the 

address specified in the instruction A n, by unity every time this instruc¬ 
tion A n is used. This can be done by adding the instruction itself into 
the arithmetical unit and adding unity to it in the digital position 
corresponding to the unit in n. The number consisting of unity 
in this digital position is conventionally kept in storage location or 

just this purpose. . . 

These instructions may have to be preceded by preliminary instruc¬ 
tions to clear the accumulator and location 4. Thus so far we have 
the following instructions, at addresses starting from, say, m: 


Storage 

location 

771 

7/t+l 
7/1 + 2 
7/1 + 3 
7/1 + 4 
7/1 + 5 

7/1 + 6 

7/1 + 7 



Notes 

to clear accumulator 1 preliminary 
to clear C(4) / preparation 

the addition process; the instruction in location (m + 3) 
becomes successively A 80, A 81,..., A 150^ the result 
of instructions in locations (m + 5) to (m+ ') 

those instructions increase by unity the address specified 
in the instruction in location (m + 3) 


Each time this set of operations is repeated, the address specified in 
the instruction in location (m + 3) is increased by unity. Before repea g 
it again, the machine must test that the addition is not yet complete. 
This could be done by keeping at a separate address, say location 6 
count of the number of numbers C(«) added, and examining C(6) after 
each addition to see if the requisite number of add, ions has beenIn¬ 
formed In this case, however, the address specified m the instruction 
in location (m + 3) is increased by unity for each addition perfumed so 
this instruction itself can be used as the counter There 
ways in which it can be so used; the following is not he most elegan . 

but is straightforward and probably the easiest to follow. 

If the last number added .vas C(n), then after the 
address (m + 6) has been carried out, the content of e “ cc thcn 

A ( 7 i+l). If the instruction in (m + 7) is replaced >y ’ 

A 1 will be retained in the accumulator after being trails fenr dto 
address (m+3). If then d 151 is subtracted from the_co„ten ofth 
accumulator, the result is (a-150), which is negative . k ' 
incomplete and zero if it is complete, and an E-order can hen be used 
to discriminate between the procedure necessary to repeat the proc 
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of addition and that required if the addition is complete. Thus the 
process can be completed by the following instructions: 


Storage 




location 

Content- 

Notes 

7)1 + 7 

U 

(m + 3) 

in place of T (in + 3) in first schedule 

m + 8 

S 

(7/1+12) 


7/1 + 9 

E 

(m+13) 

if the addition is complete, C(Acc) = 0 and the next 
instruction is taken from address (//i + 13); if it is incom¬ 
plete, then C(Acc) is negative, and the next instruction 
is taken from address (m+ 10) 

7/1 + 10 

T 

0 

clears accumulator 

7/1 + 1 1 

E 

(»* + 2) 

this instruction is only reached if the addition is incom¬ 
plete; 0(.4cc) = 0 as the result of the previous instruc- 
t ion, so the next instruction is taken from address (/n + 2) 
to repeat the addition process; and the accumulator has 
been cleared in preparation for this addition 

7/z + 1 2 

7/1 +1 :i 

A 

151 

the next instruction after the addition is complete 


r I lie word’ .1 151 in location (m-p 12) here does not represent an opera¬ 
tion which is carried out by the machine in this calculation; it is only 
used for the purpose of comparison with the instruction -4 n constructed 
by the machine to test whether the addition is complete or not. 

With the addition programmed in this way, and with this method of 
coding ami operating on instructions, only thirteen instructions are 
necessary to carry out the addition; and tin's number of instructions is 
required for any similar addition, however many numbers have to be added. 
this illustrates, in a simple case, that when the machine can be pro¬ 
grammed to alter its own instructions in the course of a calculation, the 
number of instructions which have to be programmed explicitly, and 
stored, does not necessarily increase with the total number of operations 
to he carried out. 

t )t the thirteen instructions in the above process, onlv three, namely 
those in locations (m-\-2), (m -f 3), and (m -f 4) are directly concerned with 
arithmetical operations on the numbers to be added; the others are 
concerned with the organization of the calculation. This use of the 
machine to organize its own work is characteristic of programs for a 
machine which provides facilities for doing operations on the instruc¬ 
tions themselves in the course of a calculation. 

In using these instructions, the number 80, which is the address from 
which the lirst term in the sum is to be taken, is lost, since the content 
of location (w + .j) has been changed in the course of forming the sum. 
If this addition process has to be repeated in the course of a single 
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calculation, the instruction A 80 must be replaced in location (m + 3); or 
better, it can be stored somewhere else where it can remain unaltered, 
and transferred to the appropriate address by instructions which form 
part of the preliminary preparation before the addition process itself is 
started. We thus obtain the instructions in the second column of the 

following table 


Storage 



Entry on 




location 

Content 


tape 



OlCS 






GK 


control combination 


VI 

T 

0 

T 


F 


1 


771+1 

T 

4 

T 

4 

F 

1 

preliminary 

• 

771 + 2 

A 

(m+14) 

A 

14 


to set C(m + 5) 

preparation 

771 + 3 

T 

(m + 5) 

T 

5 

e 1 

= A 80 



771 + 4 

A 

4 

A 

4 

F 

C(m + 5) is initially set 1 

tho 

th-4-5 

A 

n 

Z 


F 

by (771 + 3). and 

later 

addition 







modified by (rn + U) 

process 

771+6 

T 

4 

T 

4 

F 


J 


771 + 7 

A 

(777 + 5) 

A 

5 

e ' 

add 1 to the address specified in the 

771+8 

A 

9 

A 

2 

F 

A 

lion in location ( 

777 +5) 


771 + 9 
... 1 1 A 

V 

V 

(m + 5) 

( Ml 4- 1 M 

U 

s 

5 

15 

6 , 
e 

! 

test if addition complete 


m+11 
;n+ 12 

7/1+13 

771 + 1 4 
771+15 

771+16 


E (771+10) 

T o 

E (771 + 4) 
+ 80 

A 151 


E 

T 

E 

A 

A 


10 


4 

80 

151 


0 

E 

0 

F 

F 


to repeat the addit ion process if t lie addit ion 
is not complete 

stored for use by instruction in ( 771 + 2) 
stored for use by instruction in ( 7/1 + 10) 
next instruction after addition complete 

JlhUprogram the address (80) of the first of the sequence of numbers 
to be added is stored, as part of an instruction, m location m + 14). The 
instructions in locations (m + 2), (m + 3) transfer tins instruction from 
(m+14) to (m + 5) as part of the preliminary preparatron for tins ci ¬ 
tation. The initial content of location (m + 5) ,s therefore .rrclcv^ 
since it is replaced by the instruction A SO from address (« + W) u forc 
the machine reaches the address (m + 5) ... carrying out its sequence of 
operations. The stop order Z is often used as the initial eontm of a 
location such as (m + 5) here, so that, if by some mistake it ,s not placed 
by another instruction, the machine will stop there, and by so doing » 
give an indication of the mistake. 

12.6. Form of instructions on the input tape 

In this schedule some of the instructions refer to addresses 1. 
depend on the value of m, which is determined by the pos.t.on ... t 
store in which these instructions the,..selves are placed whereas otl.e . 

t The on tried in the third column aro explained in * 
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are independent of this position. But in drawing up the program of 
instructions, and punching them on the input tape, it is convenient to 
have them in a form which does not explicitly depend on the value of m. 
This can bo done by including in each instruction, as punched on the 
tape, an additional symbol, which for the EDSAC takes the form of a 
code letter ; F for an instruction in which the number is to be read as it 
stands as the address to which an instruction refers, and 6 for one in 
which the number punched has to have the value of m added to it in 
order to give that address. That is to say, the terminal code letter 6 
indicates that the number in the instruction is an address relative to the 
address of the first of the group of instructions as zero; this can be 
expressed by saving that this number specifies a ‘relative address’. To 
specify the address from which these relative addresses are counted, 
a special instruction, called a ‘control combination’, which is coded as 
the let ters GK, is punched on the tape immediately before the first of the 
group of instructions. This is an instruction to the machine that in 
the process of taking in entries from the tape, the address m of the location 
in which the next instruction is placed is to be recorded so as to be avail¬ 
able for constructing t headdresses in those instructions which are punched 
on the tape with the code letter 6. 

Thus on the tape the instructions for placing |C(6)| in location 4 

(§!-•'> («)) take the form shown in the fourth column of the following 

schedule: 


Relative Entry on 


Address 

Content 

address 


tape 






GK 


7fi 

A 6 

0 

A 

6 

F 

;/< + 1 

E (m + 4) 

1 

E 

4 

0 

m -f 2 

T 0 

o 

T 


F 

/// + 3 

.S’ 0 

3 

S 


F 

in -f 4 

T 4 

4 

T 

4 

F 


1 he first t wo columns are the same as in the schedule on p. 2G3; the fourth 
shows the way in which the instructions are written and punched. 
Similarly the instructions for the second example take the form shown in 
the third column of the schedule on p. 267. The code letters GK, F, and 
0 do not appear in the instructions in the form they take in the store; 
they are subsidiary instructions to the machine regarding the interpreta¬ 
tion of the punchings on the tape and the formation of contents of storage 
locations from them in the course of the input process. The code letter 
F or d also serves to identify the end of the number which specifies the 
address to which the instruction refers. 
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As thus coded on the tape, the instructions have the same form when¬ 
ever they are placed in the store, and once drawn up they can be copied 
as they stand whenever the calculation they carry out is required. This 
copying can be done and checked mechanically. An important extension 
of this use of code letters enables a set of instructions to be punched in 
such a form that they can be used for the calculation of 

I C(j) to j 3 

f-j X 

for any predetermined values of the parameters j v j 2 , and j 3 , that is to 
say, values which are known before the calculation is started, and do 
not depend on the results obtained in the course of it. 

Similar facilities could be provided in other ways if the control system 
of the machine were designed appropriately. For example, the operations 
called for by the code letters F and Q in the above example could be 
carried out not in the course of input of the instructions but by an 
auxiliary operation earned out on each instruction in the process o 
transferring it from the store to the control unit, immediately before 
the operation specified by it is carried out. 


12.7. Sub-routines 

There are two kinds of contexts in which sub routines may be used. 
In the first, the sub-routine is either required only once in the course of 
a calculation, or it is always required at the same stage of a repetitive 
calculation, so that the instructions which immediately precede and 
follow the sub-routine are at definite addresses m the programme A 
sub-routine suitable for such a context is called an open sub-routine. 
The sequence of instructions considered in the previous section can be 
regarded as an open sub routine for the process 


t C*(80-f-j) to 4. 
o 

In operation, it is entered from an instruction in location (m-l) and 
followed by one in location (m+16). The instruction E m +16) in 
location (m + 11), which results in the next instruction being taken from 
location (tw+ 16) when the addition process of the subroutine is complete 
is usually called the ‘link’ between the end of the sub routine and 

m t .. 

tione .1.1.1 immediately proed. or ™ 

located at different addresses on the different occasions on u 
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used, then this form is not convenient, as the sub-routine cannot be 
placed in the store so that its first instruction is always next after the 
instruction which immediately precedes it in time; and, more important, 
the address to be used in the ‘link’ instruction, for return to the main 
program when the process carried out by the sub-routine is complete, 
cannot be punched on the input tape as it is different on the different 
occasions on which the sub-routine is used. 


In these circumstances it is more convenient to use another form 
of sub-routine, called a ‘closed’ form, which can be placed in the store 
in any position, independently of the main program, and which 
includes instructions for forming the link instruction for return to the 
main program. How this is done depends on how the main program 
is planned and how the sub-routine is entered from the main program. 

The standard method used with the EDSAC will be illustrated first 
by a simple example. The sequence of operations for placing |C(6)| 
m location 4 (see § 12.5 example (a)) is too short to be worth coding by 
itself as a sub-routine, though it might well form part of a longer sub¬ 
routine; but for the purpose of illustration let us consider a closed sub¬ 
routine for this process. 


Let n be the address of the beginning of the sub-routine; it is to be 
entered after the instruction in location (»i-l) has been carried out, the 
accumulator being then clear. It is entered as follows: 


-1 t/'Jrcss 
m 

m I 

>/*+ 2 


Cot i ten l 

A m 
(S n 


Entry on Holes 

tape 

-•1 m F accumulator initially clear 
a n F the function digits representing A have a 1 
in the sign-digital position; hence the next 
instruction is taken from address n, the 
beginning of the sub-routine 

next instruction after completion of sub¬ 
routine 


The instruction .4 m adds itself into the accumulator, so that when 
the sub routine is entered, the accumulator contains a record of the 
pomt in the main programme from which it was entered. The link 
instruction for return to the main programme must be E (m + 2). yhis 
is obtained by adding to A m, which is the content of the accumulator 
on entering the sub-routine, such a group of digits, or ‘word’, that it 
converts the word Am into the ‘word’ E (m+ 2). The ‘word’ which 
is required for this purpose is conventionally stored as the content of 
location 3 for use in just this context. Thus the instruction ,1 3 results 
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in the link instruction being formed in the accumulator, from which it is 
transferred to the location immediately following the instruction for 
the last operation of the sub-routine, which is arranged to leave the 

accumulator clear. 


Address 

n 


»i +1 
n + 2 
n + 3 
n + 4 
n + 5 
n + G 
n + 7 


Relative Entry on 


Content 

address 

tape 

GK 

A 3 

0 A 

3 F 


T 

n + 7 

1 

T 

7 

e 

A 

(i 

2 

A 

G 

Fy 

E 

n + 6 

3 

E 

G 

6 

T 

0 

4 

T 


F 

S 

0 

5 

S 


F 

T 

4 

6 

T 

4 

F, 

Z 


7 

Z 


F 


Notes 

control combination (on tape only) 
entered with C(Acc) = A m F. This 
instruction results in the link instruction 
E (m4-2) being formed in the accumu¬ 
lator 

plants link in (n + 7) 

10(G) | to location 4 (as in § 12.5, example 

(«» 

C(»i + 7) becomes E (tn + 2) as the result 
of instruction in location (n + 1) 


The method here given for entering a closed sub routine, forming 
the link instruction, and planting it at the appropriate place in the sub¬ 
routine is a general one; the only change required is that the instruction 
T 7 0 in this example is replaced by T r 6. where r is the relative address, 
within the sub-routine, of the location for the link instruction. 

This method of using closed sub-routines can be developed to include 
the possibility of modifying the instructions of the sub-routine itself 
in the course of the calculation,t so that, for example, a single sub-routine 
for the process |C(j)l to location k could be used for different pairs of 
values off and k at different stages in a single calculation, the appiopnate 
values of these parameters being incorporated in the sub routine on 
each occasion on which it is used. Similarly, a closed sub routine based 
on the process of § 12.5, example (b), could be used for any calculation of 

the form b 

2 C(j) to c, 

where «, b, and e may have different values for different uses of this 
same sub routine within a single calculation. 


12.8. Hand and automatic calculation 

Almost any method for doing a calculation by hand, that is, wit i a 
desk machine but without the use of an automatic macluue, can be 
programmed for an automatic machine. A possible exception 

t See, for example, D. J. Wheeler. Proc. lloy. Soc. A. 202 (1950). 573. 
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relaxation process (§§8.5, 10.61) for which it would be difficult to 
formalize the judgements one uses in practice about when and by how 
much to over-relax, and when to use group relaxations, and to express 
these judgements in terms of operating instructions to an automatic 
machine. But it does not follow that the best method for a hand cal¬ 
culation is the best method for an automatic machine. There are three 
main reasons for this difference. 

First, in most hand calculations of any magnitude, the time taken in 
carrying out the numerical work is substantially longer than the time 
taken in planning it, whereas with an automatic machine the time taken 
to carry it out may be shorter than the time taken to program and 
code it. Thus in a hand calculation it is worth spending some time in 
planning the calculation to save numerical work, whereas with an 
automatic machine it may be best to obtain the same results by a simple 
process involving a large number of steps to save the time that would be 
taken in planning, programming, and coding a less simple method using 
fewer numerical steps. For example, on an automatic machine a 
relatively large number of repetitions of a simple first-order iterative 
process (§9.3) may be preferable to a smaller number of repetitions of a 
more complicated second-order process. And in calculating an integral 
as a function of the upper limit, it might be best with an automatic 
machine to use a very simple integration formula, such as Simpson’s 
rule or even the trapezium rule, with a large number of short intervals, 
<>•01, or 0 005, or even perhaps 0-001, when in a hand calculation one 
might prefer to use an integration formula to sixth or eighth differences 
of the integrand, with interval 0-1. 


Secondly, the storage capacity of a machine is limited, whereas that 
°f 1 he working sheets of a hand calculat ion is practically unlimited. This 
has several reactions on programming for an automatic machine. For 
example: (i) use of many repetitions of a simple procedure which can be 
programmed in a few instructions is preferable to a few repetitions of a 
more elaborate procedure for which the longer program would take 
more storage space; (ii) a strictly repetitive procedure is to be preferred 
to a procedure which is mainly repetitive but for which special occasional 
processes have to be used in addition: for example, in some processes 
foi the numerical integrat ion of different ial equations a special procedure 
is needed for the first interval of the integration; the instructions for 
this special procedure will take up some storage space but will be used 


once only for each solution, and a method which does not require a 
special starting process may be preferred; (iii) it may be preferable to 
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calculate values of standard functions, such as circular and exponential 

functions and their inverses, as they are required, rather than to store 

tables and the instructions for entering them and interpolating in them. 

And thirdly it is usually no shorter or easier to calculate with simple 

numbers than with numbers of many digits. For example, if e v is cal- 

1 

culated from a series, then in calculating, say, J e x 'dx by a Gauss formula 

o 

(see § 0.55) the fact that e 1 * is required for values of x such as 0-230765 
(x 2 for a five-point Gauss formula, see § 6.57) and not only for simple 
values is no drawback. 

All these differences have considerable influence on the choice of 
methods for carrying out calculations by automatic machines. For 
example, for evaluating an integral between fixed limits formulae of the 
Gauss type are much more attractive for work with an automatic 
machine than for a hand computation. Also for evaluating an integral 
as a function of its upper limit it might even be better to do a number of 
independent integrations by means of a Gauss formula, with different 
values of the upper limit, rather than to build up the integral by 
accumulating a sequence of contributions to it. And for the solution of 
partial differential equations of elliptic type, a form of the Richardson- 
Liebmann process (§10.64) may be more convenient for an automatic 

machine than the relaxation process. 

Some work has been done on the development of methods particularly 
suited to the capabilities and limitations of automatic machines, but 
the main developments of this branch of numerical analysis probably 

still lie in the future. 
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EXAMPLES 

Note : Several of the following examples are specimens of types of which the reader 
can make up other examples for himself. For instance, in the first example e° 1 
could be replaced by some other number, and in Example 4, the series to be 
evaluated could be replaced by the series solution of some other second-order linear 
differential equation with the first derivative absent. 

1. Given e 01 = 1-105171 to six decimals: 

(i) Calculate y n = c°' ln upton = 10 by successive multiplication and transfer. 

(ii) Check tho results by verifying the following relations between the 

!/«X xji = y 10 , 2/io/l/7 = 2/3. 

2/7 X 2/2 = 2 / 9 . 2/w/l/s = 2/i» 

2/6 X 2/3 = 2/s- I/a/'/a = 2/ 2 - 

(iii) Check tho results by differencing the values of y n (including the value 
?/ 0 = 1) to second differences, and verifying that (8 2 y„)/ y n is constant. 

(Note: The main purpose of this example is to give practice in the uso of a desk 
machine; the method of checking in section (ii) of the example is not recommended 
as a standard procedure for regular use.) 

2. (liven < nm = 1-133148, evaluate 2[(cosh 0-125)— 1] without writing down 
anv intermediate results. 

Prove the relation 8-’c r = 2[(cosh S.r) — l ]c x and use it to build up c° 125n up to 
n — 10. 

3. Given sin 10° = 0-1736482, find (1— cos 10°) to seven decimals by an iterative 

process based on tho formula 

1 — cosx = sin 2 x/[2 —(1 — cosx)]. 

Prove the relation S : (sinx) = — 2( 1 — cos 8.r)sinx, and use it to build up a table 
of sin(n . 10) ns far as n = 9. 

4. Evaluate y = 4 * 7 

to five decimals for x — 0(0-2)2 0, keeping seven decimals in the individual terms 
and rounding off tho sums to five decimals. 

Check the results by evaluating y" from the differential equation y" = 1-f.r?/ 
satisfied by this function y, and verifying the relation 

5 2 yj = (3x0 3 [2/;+A8=2/;-rlo8 4 2/;j + O(8x)». 

6. Show that the function y = (sin.r — xeosx)/x satisfies the differential equa¬ 
tion j/"4-( 1 — 2Jx 2 )y = 0. 

Evaluate this function to five decimals, for x ~ 0(0-1 )2-2, from its power series 
expansion, and check the results by uso of the differential equation. 

6. Calculate 2on a desk machine for = 1-274,.r, = 0-984,x 3 = 1-577, 

n In 

x, — 0-126 without writing down any intermediate results. 
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7. Find 3 X (£3. 8s. 7d.) +19 X (£1. 17s. 9d.) + 16 x (£2. 2s. 11<7.) using an ordinary 
desk machine, setting the sterling amounts in £. s. d. and exhibiting the results in 
£. s. d., without writing down any intermediate results. 

(Note: Assign the three right-hand places on the setting levers or keyboard to 
pence, the next three to shillings, and the rest to pounds. After forming the sum, 
reduce the number of pence by a multiple of 12, and add that same multiple to the 
shillings, by adding 988 in the right-hand three pluces until the number of pence 
in the result is less than 12. Treat the shillings similarly.)t 

8. Build up the cubic f(x) = x 3 - 5x* + Gx + 1 between x = 2 and i = 3 at 
intervals of 01 by means of a difference table. From these results estimate the 
position x m and magnitude of the minimum off(x) near x = 2-5. \ erify by solving 
the quadratic for .r w and evaluating/(x m ). 

(Note: Change to £ = x— 2 as variable, verifying the transformed form of the 
cubic by evaluating it for £ = 0, ±1, ±2 and comparing with the values of/(x) 

on p. 43.) 

9. Evaluato 0-623x 3 - l-876x 2 + 5-623x+2-875 to three decimals for 

x = 0(0-32)2-56 

and check the results by differencing. 

10. The following values are alleged to be copied from a table of xK Locate and 
correct the mistakes by examination of the differences. 


X 

f(x) 

X 

f(x) 

27 

3-00000 

35 

3-27107 

28 

•03059 

36 

•30193 

29 

•07232 

37 

•33332 

30 

•10723 

38 

•36198 

31 

•14318 

39 

•39121 

32 

•17480 

40 

•41995 

33 

•20753 

41 

•44852 

34 

3-23961 

42 

3-47603 


11. Using 6-figure tables of sinx°, calculate tho function 

y = s i nx” + 2.10-.[f^^]-p[-(x- 6 0,VI00] 

for x = 30( 1 )70, and round off to five decimals. 

Compare tho second and fourth differences of the rounded values of y with those 

of fivc-figuro values of win# 0 . 

Repeat for x = 25(5)75 and for x = -20(10)120. 

(Note: y-ainx° can bo regarded as an ‘error’ in a table of sinx . 1■ »© I ,ur I " 
this example is to illustrate that smooth differences do not necessarily imply 
dom from error; tho difference., of y nt interval, 1 in * are no more irregular him 
those of sin X. It also shows that a table may appear smooth on a an, seao 
represented by tho differences at a small interval of *, but unsmooth on a la.to 

scale.) 

t This procedure for using a decimal machine for certain calculations in sterling was 
uhowii to mo by Dr. L. J. Comrie. 
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12. Show that the sum £ 8 ! / 2 ;>i , of alternate second differences, can be expressed 

i=o 

in terms of the operator U = ( 8x)d/dx as (tanh if ’)(A/ — /o)- 

Deduce the value of this sum when/ is a periodic function which is an even func¬ 
tion both of (x—x„) and of (x—x w ), and examine whether or nut this result is 
independent of rounding errors in the/ values. 

13. From the table of the function y calculated in example 4: 

(i) Use the ‘half-way’ interpolation formula to obtain// at x — 0-7, 0-9, 1-1, 1*3. 

(ii) Interpolate y for x = 0-95(0 01)1 00 by Everett’s formula. 

(iii) Find the value of x for which y = 0-5 

(a) by inverse interpolation using the values at 0-1 intervals only; 

( b) by inverse interpolation using the values at 0-01 intervals calculated 
under (ii). 

14. From a table of sinx at intervals of 10° in x (see p. 01): 

(i) Find sin 23° 20' and sin 26° 40'. 

(ii) Find sin -1 0-40 

(а) by inverse interpolation using a formula involving the differences of 
sin x as a function of x; 

(б) by using Lagrange’s interpolation formula, treating x as a function of 
sinx, and verifying by interpolating in the table of sinx for the valuo 
of sin -1 0-40 obtained ; 

(c) by using the divided differences of x as a function of sinx. 

15. Construct a table of values of log(n!) to five decimals for n = 5(1)12. Use 
this table to interpolate log(x!) for x — 8.V and 9J, and verify that the interpolated 
values satisfy tho relation (9.J) !/(8.l)! = 9.J. 

Derive a value for (1)! = Is'tt, and hence a value of n. 

16. Given the values 

x = 0 1 2 3 4 5 

y = 0 1 8 27 64 125 

examine the result of attempting to interpolate x for y = 20 by a six-point Lagrange 
formula for x as a function of y. 

If the calculation were done by the use of divided differences what, symptoms 
would suggest that the result should be accepted with suspicion? 

17. Continue to x 0*80 the subtabulation started in the example in §5.6*1. 

X 

18. Continue to x = 10, by intervals x -- U U5, the evaluation of J e ,r * dw 
started in the example in § 6.4. 

19. Show that 

j /(■»■) d* - 4(Sx)[l-H8 2 +i, 7 0 -S«—i--I- 0(8x)“ 
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20. Show that if for integration over a given range x — a to x = b, this range 
is divided into 2 J equal intervals 8x = (b—a)/2J, then 

f /(x) dx = ((6—cr)/J][ 2 / 2/+1 + 0 .2 .2 + 

J l > = 0 J => 0 ) — V 

a 

+ T*T+ 

tho sums being sums of alternate values of t\xc function and of its differences of 
even order. 

(Note: For the evaluation of an integral over a given range this formula, taken as 
far as the 8'-/ term, is probably the most convenient form of Simpson’s rule for 
practical work; higher-difference terms can easily be included if appreciable.) 


21. The function erfex is defined by 

00 

erfex = (2 /tt*) \ e~^ dw. 

Z 


Evaluate erfex for x = 0(0 1)1 2 to five decimals by quadrature. Evaluate 
2 ferfe wdw for x = 0(01)10 by quadrature and check by use of the relation 


2 J erfet vdw = (2/77*)e-**- 2x erfc x. 

(Notes: (i) erfcO = 1; 2 jertewdw = 2/rr* = 1 128370; (ii) the relation between 

0 

ferfe wdw and erfex is obtained by integrating by parts.) 


22. Evaluate f l<r*/(x+\)\dx and W + .)]<* to five decimals by 

quadrature. Check by evaluating the difference J [e~*'l(x+ l)(x+2)] dx between 
these integrals by an independent quadrature. 


23. Evaluate y = (l/») J cos(xainfl) M, to four decimals, by quadrature for 

at - 0(i„)15rr. Estimate. a°s closely as you can from the results, tho smallest 
positive value of x for which y = 0. 

24. Continue to a =1-6 the integration of the equation y = 0 -*')» slorlc ‘ l 
in the worked example in § 7.2. 

25. Continue to x = 20 the integration of the equation y’ = 1 - 2 ^ -staitc.d m 

the example in § 7.3. The solution of this equation is y = c 1 ]' du Co,T 1 

the results of the integration of this differential equation with those of the worked 
example in § 0.4 und its continuation in Example 
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26. The function/(x) = j e~ u 'l(x+u) du satisfies the differential equation 

o 

f'-\-2xf = —(see §6.56). 

X 

Starting from the value of/(l) obtained in Example 22, integrate this equation as 
far as x = 2. Compare the value of/(2) obtained by integration with the value 
of/(2) obtained in Example 22. 

\-TT 

27. Evaluate y = (3/rr) J e ixeo,$ dd to four decimals for x = 0(1)2 

o 

(а) by expanding the integrand in series, integrating term by term with respect 
to 6, and evaluating the resulting series in x; 

(б) by quadrature; 

(e) by obtaining a second-order different ial equation satisfied by y as a function 
of x, and evaluating the appropriate solution by numerical integration. 

28. Find, to two decimals, the solution of the equations 

lSx — 4y-f3c = 53, 

10x+16y + 2s = 87, 

5x-f 3y-J- 9c = 21 

(«) by elimination; (h) by relaxation. 

26. A cubic y f/ 0 .r :| « 2 .r-f-a 3 takes the values y — 12, 6, 0, 12 for 

x — 2, n, 1, 3 respectively. Find the values of the coefficients a 0 , a lt a,, a 3 

(i) by substituting .r = —2, 0, 1, 3 and solving the resulting simultaneous 
equations for the coefficients; 

(ii) by use of divided differences (seo §5.72). 


30. U.-e Milne's method (§ 6.7) to obtain an expression for the error term of tho 

integration formula 


| /(•>•) < 1 * 
*0 




31. (i) Invert the matrix 



(ii) 1* ind the eharaeterist i«* values and characteristic vectors of this matrix. 


32. Construct the in verse of t ho matrix 


I —23 11 

11 — 3 

\ I 



from its characteristic vectors ami tho reciprocals 
determined in §8.03. 

(Xotc: This is the matrix whose inverse is found 
example in § 8.3.) 


of its characteristic values as 
hy elimination in the worked 
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33. Solve the equations 


14x 1 -f7*2+ 17a: 3 + 8 *4 

7 *!+11*3+1 3 ar 3 + 4 :r* 
17*!+13x 2 + 42x3-11*4 

8*! + 4x 2 — 11 x 3 + 30x 4 


134, 

70, 

77, 

70. 


34. Find the solution of the equation y' = x'-y- l for which y = 0 at x = ± 2 

(i) by evaluating a particular integral and a complementary function by stop, 
by-step integration and forming the appropriate linear combination, 

(ii) by a relaxation method. 

35. Work out a relaxation method for finding a solution of V-l = 0 for a ^tcm 
with symmetry about an axis. Apply it to find an -VP~tcso ution of V ^ - 0 
for the axiaUy-symmetrical system formed by rotating 1 »g. - (Pr¬ 
axis of symmetry. 

y <:,^e LIsVo «“t mot a: = 0°; 

InTl,: planed I £ V - O^FmdL two decimals the variation of V in the space 
between the planes. 

37 V satisfies V’F = 2/o= inside a square of side a, and 1' — Oon the boundary 
of the r q uam fL to three decimals the value of at the centre of the square. 

38. Find to two decimals the other roots of the simultaneous non-linear equations 
of which one root is found in § 9.0. 


39. Use the iterative formulae 

(a) 2/n+i = ifi/i» + ( a /2/ii)]» 
for a* to evaluate V5 and V60. 


(b) l/n+l = l J< i< 3<< 


40. Show that the formula 

2/n + i = 2/ n tP+ 1 ” a ^ 
gives a second-order iterative process for 1A* 1/p - 

EKfA- 


(l/9a)>/». Use it to find (8/9)*/® and (1/9)*/® 
that the ratio of the results is 2 1<3 . 
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Accumulator of desk machine, 11, 15, 49, 
61. 

Accuracy, 9, 172, 248. 

Adding machine, 11, 19, 43. 

Addition on desk machine, 12, 14. 

— of complex numbers, 214. 

Airy functions, 77, 119, 238. 

Aitken, A. C., 84. 

Algebraic equations: linear simultaneous, 
150-77; non-linear, in one variable, 190- 
207; non-linear, in two or moro vari¬ 
ables, 207-13. 

Analysis of observations, 4, 243. 
Automatic digital calculating machines, 
25, 255-73; accumulator of, 257; arith¬ 
metical unit of, 255; control in, 255, 
258; instruction in, 256, 257, 262; pro¬ 
gram for, 256; programming, examples 
of, 263; scriul storage in, 257; store of, 
255; sub-rout ine for, 258, 269; word in, 
257. 

Automatic division on desk machine, 19. 
Auxiliary function in smoothing, 252. 

— variables in tables; 22. 

Averaging operator p, 39, 53; inverso of, 
63. 

Back-substitution, 159, 162. 

Bernoulli numbers, 99. 

Bessel functions, 28, 170, 286. 

— interpolation coefficients, 69, 70. 

Bessel's interpolation formula, 65, 68-74, 

87, 91, 92, 96. 

Bickloy, W. G., 56, 180, 216. ( 

Boundary conditions, 125, 220, 222. 
Bridging differences, SO. 

Calculating machines: automatic, 24, 255- 
73; desk, 10, 11, 25, 26, 28. 

Characteristic values: of matrices. 153. 
154, 178-89; of ordinary differential 
equations, 146, 189; of partial differen¬ 
tial equations, 231. 

— vectors of matrices, 178-88. 
Characteristics of hyperbolic partiul differ¬ 
ential equations, 236-8. 

Checking and checks, 3. 5, 29, 30, 31, 40, 

43. 44. 45, 63, 74, 81, 83. 84, 94, 103, 104, 
127, 128. 129, 131, 138. 139, 157, 158, 
166, 170, 182, 186, 187, 209, 215, 240, 
242, 246, 248. 

Choleski method, 164. | 

Clearing, 11. 


I Circular functions, 33, 35, 63, 79, 273. 
Complement, 15. 

Complex numbers, 214, 215. 

— variable, functions of, 214, 218. 
Comrie, L. J., 65, 68, 71, 75, 76, 84, 92. 
Continued product, 28. 

Convergence, 3, 171. 

Critical tables, 21. 

Cross-sum check, 157, 158, 160, 161, 163. 
Cubic equation, 200. 

Cylindrical coordinates, 219. 

Dcdekind section, 3. 

Deferred approach to the limit, 139. 
Derivatives and differences, 50, 51, 56-59. 
Determinant, 150, 152, 161. 

Difference operators, 37, 51-59; inverses 
of, 53. 

Differences, finite. 36-59; building up 
from, 42, 49; checking by. 40, 44; and 
derivatives. 38. 50, 56-59, 172; effect of 
errors on, 37, 44-47 ; in terms of function 
values, 39, 54 ; notation for. 38 ; smooth¬ 
ing by, 252. 

— baekwnrd, 38. 

— central, 38, 53. 

— forward, 28, 52, 64, 102, 110, 240. 

— leading, 28. 

Sec also Differentiation, Integration, 
Interpolation. 

Differential equations, 1, 2, 107, 108, 114, 
125, 213; see also Ordinary differential 
equations, Partial differential equations. 
— operator, 52. 55, 56, 216; inverso of, 56. 
Differentiation: formulae, 56, 118; graphi¬ 
cal, 120; numerical. 110-20. 

Diffusion equation, 221, 233. 

Digital machine. 255. 

Direct interpolation, 60. 

Divided differences, 87. 

Division on desk machine, 17. 

— of complex numbers, 214. 

— of a polynomial by a quadratic, 201-4. 
Doodson, A. T., 251. 

EDSAC, 261. 

Elimination, 155-62, 207. 

Elliptic partial differential equation, 172, 
222, 223 *>37 

Errors: see Rounding errors, Systematic 
errors. Truncation errors. 

Eulcr-Maclnurin formula, 99, 102, 111, 
241, 244. 
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Euler’s transformation of a power series, 

24 °. 

Everett interpolation coefficients, 68, /l. 
Evorctt’s interpolation formula, 65, 67, 
70-75, 80, 82, 87, 96. 

Exponential extrapolation, 31, 91, 106, 
196. 

— function, 28, 77, 273. 

Factorial polynomials, 41, 66, 67, 69. 

Finite differences: in cylindrical coordi¬ 
nates, 219 ; in two dimensions, 215, 272 ; 
see also Differences, finite. 

Flow diagram, 259. 

Formulae: evaluation of, 4, 27; signifi¬ 
cance of, in numerical work, 27. 

Forward differences, 38, 52, 110, 240; in 
interpolation, 64; in integration, 102. 
Fox, L., 142, 166, 172, 230. 

Frequency analysis, 4. 

Function, continuous, 34. 

— of a complex variable, 214, 218. 

— of two variables, 214-38. 

Gamma function, 34. 

Gauss integration formula, 115, 273. 

Gibbs phenomenon, 244. 

Goodwin, E. T., 112, 113, 114, 142. 
Graphical methods, 120, 190, 208, 209, 211, 
252. ' 

Gregory integration formula, 110. 

Group relaxations, 170, 174. 

Guarding figures, 6, 249. 

/^-extrapolation, 140, 233, 234, 249. 
Halfway interpolation, 62. 

Harmonic analysis, 4, 243. 
Heat-conduction equation, 221, 233-6. 
Hollerith machines, 25. 

Hyperbolic functions, 28. 

— partial differential equations, 222, 

236-8. 

Ill-conditioned equations, 152, 161, 163, 
173, 187. 

Indeterminate forms, 31. 

Initiul conditions, 25, 222. 

Input in uutoinutic machine, 256. 

Integral conditions on solution of u differ¬ 
ential equation, 126. 

— between fixed limits, 95, 109; by solu¬ 
tion of a differential equation, 114; as 
function of upper limit, 95, 101-6. 

— equation, 4. 

— purumotric, 113. 

— twofold, 108. 

Integrating factor, 147. 

Integration formulae, 58, 59, 96-102. 


Integration, numerical, 4; of a given 
function of i t 95-116; of an ordinary 
differential equation, 95, 125-49; of u 
partial differential equation, 223-38. 

Interpolation, 4, 25, 60-94; linear, 60; 
non-linear, 23, 60, 64-92; in complex 
plane, 219. 

Interval length, cliunge of: in quadrature, 
105; in integration of differential equa¬ 
tion, 130, 176, 228. 

Inverse interpolation, 60, 61, 71, 76, 90, 
190. 195, 208. 

— of a matrix, 7. 162. 167, 173. 180. 

— operators, 53, 56, 63. 

Iterative process, 191, 272; for algebraic 
equation, 191 ; for characteristic values 
of matrix, 180 ; for differential equations, 
141, 144, 174 ; for inverse interpolation, 
91, 196; for quadratic factor of poly¬ 
nomial, 205. 


Jury problem, 126. 

Lagrange integration formulae, 111. 

— interpolation coefficients, 75. 

_interpolation formula: equal intervals 

of argument, 74-76; for inverse inter¬ 
polation, 92-94; unoquul intervals of 
argument, 83, 8 /. 

Laplace'* eipiation, 214. 223. 225. 229. 244. 
Lapluciun operator: in two dimensions. 

214. 217; in three dimensions, 219. 
Latent "mots (of inatrieea). see Character- 
ist ic values. 

_ vectors (of matrices), see Characteristic 
vectors. 

Leading differences, 38. 

Liobniunii, H.# 232. # 

Liebinann’s process (for Laplace s ecpm- 

tion), 232. 

Limiting process, 35. 

Linear cross-mean, 84. 

- ilill. reiitiiil cipiations, .-re Ordniurj. 
partial dittciential equation-*. 

— interpolation, 00. 

— operators, oil. 

— simultaneous c,nation#, •'<* h " ,,ul 
tunoous equations. 

Lower triangular matiix. 152, 104. 

Madelung, L-. 148. 

— transformation, H’b 

Marching problem. 1 

.. | | r,i, IV* Mil; claim-tci isle 

Matrices, 4, LiO. I-*-. "" • , 

values of, 153. ITS HU ; inverse. -o. -• 

107. 173. 180. 257 ; inversion ol. l-»- 
JOT’. 173, 1 SO. 

Mean differences, 39. . 
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Michel, J. G. L., 65. 

Miller, J. 0. P., 48. 

Millman, J., 133. 

Milne, W. E., 85, 94, 121. 

Mistakes, 5, 7, 28, 45, 46, 104, 158, 195, 
267. 

Modified differences, 71. 

Morris, J., 154. 

Multiple roots of algebraic equations, 197, 

201 . 

Multiplication on desk machine, 12, 16. 

— of complex numbers, 214. 

‘National’ calculating machine, 24, 78. 
Neighbouring roots of algebruic equation, 
197. 

Newton’s for ward-difference interpolation 
formula, 64. 

— iterative process for square root, 193. 
Newton-Raphson iterative process, 194, 

195. 

Nominal accuracy, see Precision. 
Non-linear interpolation, 60-92. 

Normal equations, 155. 

Normalization of characteristic vectors, 
179. 180. 

Nairn rov, 13., 133. 


Diver, F. W. J., 142. 

(tperntions table, in relaxation calculation 
1 OH, 171, 189. 

Order of an iterative process, 192. 
Ordinary differential equations, 2, 24, 125- 
1.2, 259; boundurv conditions for, 

I - .. 


first -or li. r, 133, 135, 147. 

• linear, 23, 126, 133. 136. 147, 14S. 

— second order, first derivative absent, 
126-33, 173; first derivative present, 
130. 


linear, relation to .simultaneous 
algebraic equations, 173. 

'— third and higher orders, 137. 
Orthogonality of characteristic vectors of 
ft symmetrical matrix, 178, 182 183 
180, 187. 


Parabolic partial differential equations 
232, 248. 

Partial differential equations, 221; boun 
litions for, 222. 

— elliptic, 172, 222, 223, 237. 

— hyperbolic, 222, 236, 237. 

— parabolic, 222, 232, 237. 

Purtiul fractions, 84. 

Pivotal coefficient, 157, 159, 161. 

— equation, 157, 159, 162. 


Pivotal value, 78, 79, 80. 

Poisson’s equation, 221, 227, 228. 

Polar coordinates, 214, 219. 

Polynomial equation, 198, 200. 

— divided differences of, 88; derivatives 
of, 89; evaluation of, 29, 42. 

Power series, 30, 33. 

Powers-Samas machines, 25. 

Precision, 9. 

Processes, numerical, 1, 3, 27. 

Proportional parts, in interpolation, 62. 
Punched-card machines, 25. 

Purification process, for characteristic 
vectors, 182. 

Quadratic equation, 32, 199. 

— factor of polynomial, 203. 

Quadrature, 95-116. 

Quartic equation, 200. 


Rayleigh’s principle, 180. 

Recurrence relation, 28, 172, 174, 246. 
Reduced derivatives, 77, 138. 

Relaxation method: for ulgcbraic equa¬ 
tions, 167-73, 236, 272; for character¬ 
istic vectors of matrices, 188; for ordin¬ 
ary differential equations with two-point 
boundary conditions, 173; for elliptic 
partial differential equations, 224-32. 
Residuals, 151, 155, 162, 164, 168, 173, 
224. 

Resistance function, 250. 

Riccnti transformation, 148. 

Richardson, L. F., 125, 140, 183, 232. 
Richardson-Licbmann process, 232, 273. 
Root-squaring, 200. 

Rounding errors, 5, 46, 47, 161, 163, 197, 
198, 234, 235, 246-9 ; sec also Tolerance. 
Round-off, 5, 43, 184. 

Rule of false position, 194. 


Second differences, 36, 127; direct evalua¬ 
tion of, 48; building up from, 49, 83, 
12 ( ; see also Differences, finite. 

Second-order iterative process, 192, 196 
205. 

Separation of variables, 221. 

Senes, evaluation of. 30, 33, 239. 

Shift operators, L\ bJ~ l , 52, 240. 

Short cutting. 17, 18. 

Simpson’s rule, 51, 100, 111, 219, 272; cor¬ 
rection to, 100, 219. 

Simultaneous algebraic equations: linear, 
4, 150-73, 223; non-linear, 4, 207. 

Singularity, 106, 220; integration in 
neighbourhood of, 106, 107, 113. 

Slide-rule, 11, 23, 28. 
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Smoothing, 4, 40, 247. 

Smoothness, 35, 48, 247. 

Southwell, R. V., 167, 170, 216, 230. 
Spencer, J., 250. 

Square root, 193; of complex numbers, 
215. 

Statistics, 3, 4. 

Staton, J., 114. 

Sub-tabulation, 4, 71, 77-83. 

Sub total, on adding machine, 20 r 43. 
Subtraction, on desk machine, 12, 14. 
Summation of series, see Scries. 

— operator a, 53, 101, 253. 

Symmetrical matrix, 155, 179-82. 
Systematic errors, 44, 45, 78. 

Tables, mathematical, 11, 20. 

— critical, 21. 

Tabulation, 1, 4; use of auxiliary variables 
in, 22. 

Taylor series, 50, 52, 54, 192; in interpola¬ 
tion, 77; in integration of differential 
equation, 137. 

Temple, G., 170, 180. 

Terminal conditions, 222. 


Throw-back, 71, 72, 75, 76. 91. 

Tolerance, for rounding errors, 5, 105, 110, 
119, 120, 186, 187, 199, 241. 

Total, on adding machine, 20. 

Transfer: on desk machine, 15; on auto 
matic machine, 262. 

Trapezium rule, for integration, 97, 101. 

Triangular matrix, 152, 164. 

Truncation errors, a, 94, 121, 223, 234, 249. 

Two dimensions: finite differences in, 2 la- 
20; Lapluce’s equation in, 214, 218, 223, 
225-30, 232. 

Two-point boundary conditions, 125, 143, 
172, 234. 249. 

Turing, A. M., 153. 

Upper triangular matrix, 152, 1C4. 

Wave equation, 221. 

Weber functions, 246. 

Weddle’s rule, for quadrature, 100, 110. 

Whittaker functions, 61, 240. 

Wilkes, M. V-. 133. 

Wilson, T. S., 154. 

Working sheet, 8, 25, 26, 255. 
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